-- Logs begin at Mon 2022-05-09 03:34:00 UTC, end at Mon 2022-05-09 04:41:48 UTC. -- May 09 04:11:39.198389 dv-cinder-838009-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 09 04:11:39.207446 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 9 04:11:39 2022] *** May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: nodename: dv-cinder-838009-3-iscsi May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: machine: x86_64 May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: clock source: unix May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: pcre jit disabled May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: detected number of CPU cores: 4 May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: current working directory: / May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: detected binary path: /usr/bin/uwsgi-core May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: your processes number limit is 47680 May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: your memory page size is 4096 bytes May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: detected max file descriptor number: 2048 May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: lock engine: pthread robust mutexes May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: thunder lock: enabled May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 09 04:11:39.210557 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 09 04:11:39.255584 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: Python main interpreter initialized at 0x5616642499c0 May 09 04:11:39.255584 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: python threads support enabled May 09 04:11:39.255584 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: your server socket listen backlog is limited to 100 connections May 09 04:11:39.255584 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: your mercy for graceful operations on workers is 90 seconds May 09 04:11:39.255970 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: mapped 403077 bytes (393 KB) for 2 cores May 09 04:11:39.256036 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: *** Operational MODE: preforking *** May 09 04:11:39.256312 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: *** uWSGI is running in multiple interpreter mode *** May 09 04:11:39.256312 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: spawned uWSGI master process (pid: 79752) May 09 04:11:39.256521 dv-cinder-838009-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 09 04:11:39.257132 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: spawned uWSGI worker 1 (pid: 79763, cores: 1) May 09 04:11:39.257422 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: spawned uWSGI worker 2 (pid: 79764, cores: 1) May 09 04:11:39.259998 dv-cinder-838009-3-iscsi devstack@keystone.service[79752]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 09 04:11:45.082020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 04:11:45.082020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: from cryptography.utils import int_from_bytes May 09 04:11:45.083231 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 04:11:45.083231 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: from cryptography.utils import int_from_bytes May 09 04:11:45.086498 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 04:11:45.086498 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: from cryptography.utils import int_from_bytes May 09 04:11:45.087836 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 04:11:45.087836 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: from cryptography.utils import int_from_bytes May 09 04:11:45.131615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 04:11:45.131814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 04:11:45.132055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] command line args: [] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 04:11:45.132268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 04:11:45.132446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ================================================================================ {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 04:11:45.132698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] admin_token = **** {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.132941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] config_dir = [] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.133124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.133306 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] config_source = [] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.133482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] debug = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.133677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.136095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.136095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] insecure_debug = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.136095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.136300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 04:11:45.136360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.136429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 04:11:45.136488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] list_limit = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.136548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] command line args: [] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 04:11:45.136605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] log_config_append = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.136660 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 04:11:45.136714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.136779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ================================================================================ {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 04:11:45.136837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] log_dir = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.136837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] log_file = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.136940 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] admin_token = **** {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.136996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] config_dir = [] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] config_source = [] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137293 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] debug = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137556 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] insecure_debug = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137807 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137863 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.137979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.138106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] list_limit = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.138174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.138229 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] log_config_append = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.138283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.138337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.138391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.138459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] log_dir = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.138459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] log_file = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.138569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.138624 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.138678 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] notification_format = cadf {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.138738 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.139382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.139560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.139560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.139949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.140120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.140354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] publish_errors = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.141093 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.141320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.141483 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.141715 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.141884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.142072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.142263 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.142453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.142644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.142840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.143028 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.143224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] standard_threads = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.143415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.143600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] strict_password_check = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.143785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.144039 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.144202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] notification_format = cadf {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.144440 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] use_eventlog = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.144440 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] use_journal = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.144823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.144997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] use_json = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.144997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] use_stderr = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.144997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] use_syslog = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.145371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.145371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] publish_errors = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.145663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] watch_log_file = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.145789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.145919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.146083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.146245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.146388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.146514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.146643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.146767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.146896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.147027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.147159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] standard_threads = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.147291 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.147425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] strict_password_check = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.147604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.147734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.147865 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.148056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] use_eventlog = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.148056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] use_journal = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.148300 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.148423 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] use_json = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.148546 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.148669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] use_stderr = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.148669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] use_syslog = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.148910 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.149033 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] watch_log_file = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 04:11:45.149157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.149289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.149495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.149623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.149747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.149747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.149985 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.150115 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.150249 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.150373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.150496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.150496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.150739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.150739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.150990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.150990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.151225 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.151225 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.151470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.151600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.151732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.151732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.152002 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.152128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.152253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.152403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.152540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.152680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.152841 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.152978 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.153104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.153104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.153362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.153488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.153611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.153734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.153857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.153980 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.154115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.154240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.154365 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.154499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.154629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.154758 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.154890 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.155021 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.155151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.155275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.155399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.155522 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.155720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.155848 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.155994 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.156627 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.156763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.156887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.157025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.157151 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.157280 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.157404 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.157528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.157651 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.157783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.157914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.158043 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.158167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.158291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.158421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.158556 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.158681 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.158810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.158935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.159058 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.159189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.159320 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.159444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.159444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.159676 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.159799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.159978 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.160110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.161403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.161645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.161814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.162004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.162225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.162398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.162595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.162793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.162981 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.163167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.163387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.163551 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.163806 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.164018 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.164193 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.164381 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.164602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.164766 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.164954 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.165141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.165368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.165530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.165753 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.165923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.166112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.166307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.166466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.166606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.166776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.166915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.167074 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.167212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.167339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.167464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.167588 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.167713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.167985 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.168122 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.168257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.168390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.168525 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.168658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.168784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.168909 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.169040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.169040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.169326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.enabled = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.169454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.169587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.169712 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.169837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.169961 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.170086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.170210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.170351 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.170490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.170651 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.170780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.170911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.171037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.171171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.171303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.171435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.172396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.172548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.172812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.173026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.173167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.173295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.173420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.173544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.173669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.enabled = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.173805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.173805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.174043 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.174168 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.174293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.174425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.174552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.174677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.174800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.174934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.175058 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.175191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.175317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.175442 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.175567 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.175697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.175823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.175979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.175979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.176247 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.176382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.176508 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.176634 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.176768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.176893 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.177029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.177162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.177292 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.177428 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.177554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.177684 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] auth.external = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.177819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.177945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] auth.mapped = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.178070 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.178070 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.178303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.178439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.178574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.178700 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.178825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] auth.password = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.178949 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.179073 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] auth.token = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.179197 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.179330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.179456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.179581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] catalog.caching = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.179711 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.179837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.179986 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.180146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.180297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.180434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.180561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.180686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.180686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.180927 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.181053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] credential.caching = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.181187 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.181314 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] credential.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.181439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] auth.external = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.181569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.181696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] auth.mapped = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.181821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.181955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.182093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.182219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.182344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.182484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] auth.password = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.182620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.182773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] auth.token = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.182904 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.183038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.183038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] catalog.caching = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.183362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.183503 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.183630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.183756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.183978 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.184126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.184255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.184389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.184515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.184640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.184767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.184892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] credential.caching = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.185017 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.185151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] credential.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.185278 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.185403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.185528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] federation.caching = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.185662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.185799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.185934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.186061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] federation.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.186193 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.186320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.186445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.186570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.186703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.186830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.186830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.187068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.187193 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.187859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.188194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.188331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.188458 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.188585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.188710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.188844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.189046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.189176 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.189338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.caching = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.189470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.189596 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.189721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.189855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.189983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] federation.caching = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.190121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.190247 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.190379 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.190506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] federation.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.190640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.190767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.190893 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.191018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.191144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.191270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.191406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.191533 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.191658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.191797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.191941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.192073 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.192212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.192339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.192465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.192598 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.192727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.192852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.192985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.193112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.caching = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.193245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.193402 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.193533 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.193533 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.193795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.193795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.194033 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.194158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.194285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.194418 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.194545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.194670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.194794 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.194919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.195043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.195043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.195301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.195428 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.195553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.195678 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.195678 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.195976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.195976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.196222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.196349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.196474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.196600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.196734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.196861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.196986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.196986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.197231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.197358 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.197492 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.197619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.197744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.197869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.197995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.198119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.198257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.198392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.password = **** {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.198519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.198643 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.198778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.198904 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.199038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.199165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.199291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.199416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.199542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.199668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.199801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.199944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.200076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.200202 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.200328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.200453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.200600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.200729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.200855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.200981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.201107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.201232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.201367 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.201494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.201620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.201745 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.201870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.201995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.202130 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.202257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.202391 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.202517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.202652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.password = **** {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.203047 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.203755 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.203885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.204036 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.204162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.204287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.204413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.204551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.204679 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.204805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.204940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.205065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.205190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.205323 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.205451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.205577 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.205702 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.205828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.205953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.206086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.206214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.206339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.206464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.206589 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.206724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.206858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.206985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.207110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.207235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.207360 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.207486 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.207619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.207747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.207873 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.208030 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.208158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.208292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.208419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.208554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.208680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.208805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.208932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.209065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.209192 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.209317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.209442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.209568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.209693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.209827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.209954 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.210080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] policy.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.210221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.210348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.210474 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.210609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.210736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.210862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.210988 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.211113 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] receipt.caching = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.211238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.211370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.211498 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.211624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.211749 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.211875 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.212032 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.212170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.212298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.212423 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.212547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.212683 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] resource.caching = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.212809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.212942 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.213069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.213195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] resource.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.213320 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.213445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.213570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.213704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.213840 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.213967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.214092 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.214224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] revoke.caching = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.214356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] policy.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.214559 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.214695 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.214833 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.214966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.215097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] role.cache_time = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.215228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.215368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] role.caching = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.215501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] receipt.caching = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.215642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] role.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.215773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.215955 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] role.list_limit = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.216095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.216238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.216378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.216511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.216643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.216773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.216904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.217044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.217177 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] resource.caching = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.217309 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.217441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.217586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.217717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] resource.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.217857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.217991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.218122 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.218253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.218384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.218536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.219177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.219310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] revoke.caching = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.219466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.219466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.219782 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.219782 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.220080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.220215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] role.cache_time = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.220346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.220478 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] role.caching = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.220609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.220740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] role.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.220880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.221014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] role.list_limit = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.221144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.221275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.221405 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.221545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.221678 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.221819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.221952 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.222083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.222214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.222352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.222486 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.222623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.222759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.223155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.223296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.223437 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.223578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.223710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.223852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.224043 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.224177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.224382 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.224522 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.224654 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.224785 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.225048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.225048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.225307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.225439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] token.cache_time = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.225570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.225712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] token.caching = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.225844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.225985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.226118 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.226256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] token.provider = fernet {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.226388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.226519 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.226651 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.226792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.226925 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.227056 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.227187 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.227319 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.227450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.227599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.227733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.227865 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.228020 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.228153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] trust.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.228284 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.228427 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.228561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.228693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.228826 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.228957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.229096 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.229234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.229376 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.229508 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.229641 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.229772 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.229919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.230100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.230238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] token.cache_time = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.230370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.230501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] token.caching = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.230632 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.connection = **** {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.230777 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.230921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.231066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] token.provider = fernet {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.231200 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.231343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.231476 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.231606 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.231737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.231877 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.232204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.232337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.232469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.232644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.232785 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.232961 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.233103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.233235 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] trust.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.233366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.233509 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.233641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.233782 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.233916 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.234082 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.234319 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.234456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.234658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.234843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.234980 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.235114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.235246 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.235377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.235509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.235509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.235779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.235992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.236134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.connection = **** {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.236278 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 04:11:45.236422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.236422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.236422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.236422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.236422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.236422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.236422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.236422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.236422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.237593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.237593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.237593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.237593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.237593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.237593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.237593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.237593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.237593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 04:11:45.238667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=79763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 04:11:45.249065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.249295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.249518 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.250378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.250610 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.250834 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.258911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.259531 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.288225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.288459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.290784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.291019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.342817 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.349984 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.397919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.398305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.399302 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.399538 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.466058 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.466424 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.466511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.466673 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.466885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.466969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.467107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.467327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.467406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.467542 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.467778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.468121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.491825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.492074 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.492189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.492325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.492551 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.492631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.492772 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.492989 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.493080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.493205 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.493490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.493806 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.494120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.494433 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.504805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.505259 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.519993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.520247 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.520474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.520696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.520780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.520915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.521131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.521222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.521347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79764) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.521644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.521969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.522289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.522605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.522917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=79763) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 04:11:45.539065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.539276 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 04:11:45.539615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.539819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 04:11:45.540126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.540388 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.540470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.540648 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.540817 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 04:11:45.540899 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.541310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.541613 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 04:11:45.541981 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.542338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.542697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.543063 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.552393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.552668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.552938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.553214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.555389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.555775 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.556180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.556561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.561328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.565273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.565555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.565729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.565838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.566227 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.566513 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.566778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.567050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.567323 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.567596 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.567868 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.568160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.569980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.570384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.570770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.571289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.571675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.572067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.572446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.572827 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.573209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.573586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.573961 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.595871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.597480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.597757 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.601640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.603141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.603420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.603585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.603701 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.604066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.610095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.610483 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.610866 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.611846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.612152 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.616833 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.617119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.617401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.617681 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.617955 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.618226 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.618496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.618764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.619349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.619729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.624474 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.625496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.625850 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.626234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.626613 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.627251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.627627 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.628023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.644724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.645004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.648078 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.648467 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.649551 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.653190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.653480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.653566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.657563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.657957 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.658670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.662255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.663358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.665989 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.666285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.666565 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.666840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.667113 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.667385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.667627 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.671468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.672459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.672751 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.673031 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.673305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.673578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.691346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.691721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.692005 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.692267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.693262 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.693635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.693897 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.694157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.700078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.701783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.702017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.702102 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.704104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.704371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.705374 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.705645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.705920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.706198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.706466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.707398 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.707672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.707963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.708251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.708523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.720426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.720698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.720971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.721253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.721526 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.721806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.722073 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.722361 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.722633 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.722906 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.723185 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.723458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.723724 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.724022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.742689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.742800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.743995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.745927 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.747382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.748993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.749212 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.750621 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.750894 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.751177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.751264 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.752828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.753104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.753395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.758580 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.760184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.760810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.762400 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.763375 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.763655 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.763952 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.764221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.765816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.766100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.766378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.766648 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.773514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.773800 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.774076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.775809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.776117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.776393 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.782591 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=79764) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.782859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.783133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.783406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.783680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.783974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.784243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.784514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.784777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.784862 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=79763) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 04:11:45.785042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.785124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.785302 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.785382 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.785568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.785663 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.785890 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.785973 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.786134 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=79764) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.786214 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.786377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.786630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.786902 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.787163 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.787424 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.787687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.787981 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.788249 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=79763) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 04:11:45.825770 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5616642499c0 pid: 79764 (default app) May 09 04:11:45.827221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5616642499c0 pid: 79763 (default app) May 09 04:11:45.829421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9635b90-01de-4e60-8bc6-874bb5bd81e5 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:45.829712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9635b90-01de-4e60-8bc6-874bb5bd81e5 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:45.829998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9635b90-01de-4e60-8bc6-874bb5bd81e5 None None] PATH_INFO: `/v3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:45.834185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1/1] 10.222.0.44 () {50 vars in 791 bytes} [Mon May 9 04:11:45 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:11:53.784679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ac6a2-e691-43de-9ada-fa6174d0f0d5 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:53.785184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ac6a2-e691-43de-9ada-fa6174d0f0d5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:53.785282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ac6a2-e691-43de-9ada-fa6174d0f0d5 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:53.789535 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1/2] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:11:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:11:53.794905 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654525c-f50d-4688-a787-6900a0299f4c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:53.795348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654525c-f50d-4688-a787-6900a0299f4c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:53.795466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654525c-f50d-4688-a787-6900a0299f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:53.826861 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG oslo_db.sqlalchemy.engines [None req-3654525c-f50d-4688-a787-6900a0299f4c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=79764) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 04:11:54.014161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG passlib.handlers.bcrypt [None req-3654525c-f50d-4688-a787-6900a0299f4c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=79764) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 04:11:54.014496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG passlib.handlers.bcrypt [None req-3654525c-f50d-4688-a787-6900a0299f4c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=79764) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 04:11:54.071158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-3654525c-f50d-4688-a787-6900a0299f4c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:11:54.072806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3654525c-f50d-4688-a787-6900a0299f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:11:54.196786 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 2/3] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:11:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 09 04:11:54.329557 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ac6a2-e691-43de-9ada-fa6174d0f0d5 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:11:54.363217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG oslo_db.sqlalchemy.engines [None req-1d7ac6a2-e691-43de-9ada-fa6174d0f0d5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=79763) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 04:11:54.490764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ac6a2-e691-43de-9ada-fa6174d0f0d5 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:11:54.494960 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715d747-12d7-495c-bab8-4162543701e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:11:54.495619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715d747-12d7-495c-bab8-4162543701e0 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:54.495914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715d747-12d7-495c-bab8-4162543701e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:54.496205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715d747-12d7-495c-bab8-4162543701e0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:54.504742 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d715d747-12d7-495c-bab8-4162543701e0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:11:54.556950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d715d747-12d7-495c-bab8-4162543701e0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 04:11:54.557294 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d715d747-12d7-495c-bab8-4162543701e0 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:11:54.567347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 2/4] 10.222.0.44 () {58 vars in 1324 bytes} [Mon May 9 04:11:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:11:54.573657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654525c-f50d-4688-a787-6900a0299f4c None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:11:54.588481 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654525c-f50d-4688-a787-6900a0299f4c None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:11:54.592774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5ddff-8a57-424d-85aa-23747d3ee81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:11:54.594523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5ddff-8a57-424d-85aa-23747d3ee81b admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:54.594805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5ddff-8a57-424d-85aa-23747d3ee81b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:54.595088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5ddff-8a57-424d-85aa-23747d3ee81b admin admin] PATH_INFO: `/v3/projects/528420bd736843dc87d64954f4596bbd` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:54.596613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5ddff-8a57-424d-85aa-23747d3ee81b admin admin] RBAC: Authorizing `identity:get_project(project_id=528420bd736843dc87d64954f4596bbd)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:11:54.647471 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a3c5ddff-8a57-424d-85aa-23747d3ee81b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 04:11:54.647813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5ddff-8a57-424d-85aa-23747d3ee81b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:11:54.652523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 3/5] 10.222.0.44 () {56 vars in 1207 bytes} [Mon May 9 04:11:54 2022] GET /identity/v3/projects/528420bd736843dc87d64954f4596bbd => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:11:56.855886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4eca9-bf6b-429f-b8fa-c0d2087f39ee None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:56.856420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4eca9-bf6b-429f-b8fa-c0d2087f39ee None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:56.856511 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4eca9-bf6b-429f-b8fa-c0d2087f39ee None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:56.861182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 3/6] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:11:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:11:56.866677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8de089-83c2-4709-90ca-52ffb821896b None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:56.867096 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8de089-83c2-4709-90ca-52ffb821896b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:56.867230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8de089-83c2-4709-90ca-52ffb821896b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:56.887545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-4b8de089-83c2-4709-90ca-52ffb821896b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:11:56.889084 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4b8de089-83c2-4709-90ca-52ffb821896b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:11:56.897551 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 4/7] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:11:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 09 04:11:56.919760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4eca9-bf6b-429f-b8fa-c0d2087f39ee None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:11:56.931438 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4eca9-bf6b-429f-b8fa-c0d2087f39ee None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:11:56.935499 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f4775-d7d6-4f1f-950d-af354e4ebf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:11:56.936169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f4775-d7d6-4f1f-950d-af354e4ebf94 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:56.936450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f4775-d7d6-4f1f-950d-af354e4ebf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:56.936722 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f4775-d7d6-4f1f-950d-af354e4ebf94 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:56.944323 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f4775-d7d6-4f1f-950d-af354e4ebf94 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:11:56.946064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f4775-d7d6-4f1f-950d-af354e4ebf94 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:11:56.955027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 4/8] 10.222.0.44 () {58 vars in 1324 bytes} [Mon May 9 04:11:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:11:56.960527 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8de089-83c2-4709-90ca-52ffb821896b None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:11:56.969949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8de089-83c2-4709-90ca-52ffb821896b None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:11:56.974350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0d895-f149-49c4-93ba-64fe2679f9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:11:56.975218 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0d895-f149-49c4-93ba-64fe2679f9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:56.975502 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0d895-f149-49c4-93ba-64fe2679f9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:56.975781 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0d895-f149-49c4-93ba-64fe2679f9c3 admin admin] PATH_INFO: `/v3/users/c49606bc956845408ea8db21c53f217a` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:56.977210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0d895-f149-49c4-93ba-64fe2679f9c3 admin admin] RBAC: Authorizing `identity:get_user(user_id=c49606bc956845408ea8db21c53f217a)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:11:56.979027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0d895-f149-49c4-93ba-64fe2679f9c3 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:11:56.980633 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 5/9] 10.222.0.44 () {56 vars in 1198 bytes} [Mon May 9 04:11:56 2022] GET /identity/v3/users/c49606bc956845408ea8db21c53f217a => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:11:59.261802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18704114-a001-412d-90d2-d22b8e56b4f7 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:59.262318 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18704114-a001-412d-90d2-d22b8e56b4f7 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:59.262432 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18704114-a001-412d-90d2-d22b8e56b4f7 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:59.263725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 5/10] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:11:59.269629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b181c-47f6-428e-826f-4fb618863152 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:59.270026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b181c-47f6-428e-826f-4fb618863152 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:59.270184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b181c-47f6-428e-826f-4fb618863152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:59.355310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e11a0-5dd5-4803-aca3-440a1973b7a5 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:59.355596 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e11a0-5dd5-4803-aca3-440a1973b7a5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:59.355873 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e11a0-5dd5-4803-aca3-440a1973b7a5 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:59.357266 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 6/11] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:11:59.363368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaca56-351b-4035-a6b1-82bdfb271db7 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:59.363661 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaca56-351b-4035-a6b1-82bdfb271db7 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:59.363971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaca56-351b-4035-a6b1-82bdfb271db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:59.395335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-6f4b181c-47f6-428e-826f-4fb618863152 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:11:59.396684 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6f4b181c-47f6-428e-826f-4fb618863152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:11:59.411627 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG passlib.handlers.bcrypt [None req-1dbaca56-351b-4035-a6b1-82bdfb271db7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=79763) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 04:11:59.412095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG passlib.handlers.bcrypt [None req-1dbaca56-351b-4035-a6b1-82bdfb271db7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=79763) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 04:11:59.458997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-1dbaca56-351b-4035-a6b1-82bdfb271db7 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:11:59.460613 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1dbaca56-351b-4035-a6b1-82bdfb271db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:11:59.472727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 6/12] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:11:59.495277 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b181c-47f6-428e-826f-4fb618863152 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:11:59.556611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 7/13] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:11:59.579092 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaca56-351b-4035-a6b1-82bdfb271db7 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:11:59.589791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b181c-47f6-428e-826f-4fb618863152 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:11:59.621308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb11c17-3e86-416a-9a2c-2b6cf6533a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:11:59.622093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb11c17-3e86-416a-9a2c-2b6cf6533a00 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:59.622373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb11c17-3e86-416a-9a2c-2b6cf6533a00 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:59.622642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb11c17-3e86-416a-9a2c-2b6cf6533a00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:59.633648 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb11c17-3e86-416a-9a2c-2b6cf6533a00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-efb11c17-3e86-416a-9a2c-2b6cf6533a00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:11:59.644594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:11:59.645749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:11:59.646911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:11:59.646911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:11:59.646911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:11:59.646911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:11:59.646911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:11:59.646911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:11:59.646911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:11:59.646911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:11:59.646911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:11:59.646911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:11:59.646911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:11:59.646911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:11:59.646911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 7/14] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:11:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:11:59.651459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb11c17-3e86-416a-9a2c-2b6cf6533a00 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:11:59.676895 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaca56-351b-4035-a6b1-82bdfb271db7 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:11:59.715521 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046454ec-c2a5-4e8f-b59a-9014b35e1666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:11:59.716467 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046454ec-c2a5-4e8f-b59a-9014b35e1666 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:59.716761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046454ec-c2a5-4e8f-b59a-9014b35e1666 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:59.717044 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046454ec-c2a5-4e8f-b59a-9014b35e1666 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:59.743856 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb11c17-3e86-416a-9a2c-2b6cf6533a00 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:11:59.776233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f13da5-297a-44f2-a104-a61f7885a65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:11:59.776901 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f13da5-297a-44f2-a104-a61f7885a65f None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:59.777178 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f13da5-297a-44f2-a104-a61f7885a65f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:59.777455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f13da5-297a-44f2-a104-a61f7885a65f None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:59.778200 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f13da5-297a-44f2-a104-a61f7885a65f None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:11:59.778504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f13da5-297a-44f2-a104-a61f7885a65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:11:59.779884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f13da5-297a-44f2-a104-a61f7885a65f None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:11:59.783740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046454ec-c2a5-4e8f-b59a-9014b35e1666 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:11:59.785271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046454ec-c2a5-4e8f-b59a-9014b35e1666 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:11:59.791206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 8/15] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:11:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:11:59.797104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f13da5-297a-44f2-a104-a61f7885a65f None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:11:59.877208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 8/16] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:11:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 04:11:59.882625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046454ec-c2a5-4e8f-b59a-9014b35e1666 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:11:59.894166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f13da5-297a-44f2-a104-a61f7885a65f None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:11:59.927041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2c537-327e-4e3a-a833-81d37831d546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:11:59.927924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2c537-327e-4e3a-a833-81d37831d546 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:11:59.928209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2c537-327e-4e3a-a833-81d37831d546 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:11:59.928480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2c537-327e-4e3a-a833-81d37831d546 None admin] PATH_INFO: `/v3/users/c49606bc956845408ea8db21c53f217a` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:11:59.929679 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2c537-327e-4e3a-a833-81d37831d546 None admin] RBAC: Authorizing `identity:get_user(user_id=c49606bc956845408ea8db21c53f217a)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:11:59.931121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2c537-327e-4e3a-a833-81d37831d546 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:11:59.932753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 9/17] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:11:59 2022] GET /identity/v3/users/c49606bc956845408ea8db21c53f217a => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:11:59.938674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2c537-327e-4e3a-a833-81d37831d546 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:11:59.976122 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046454ec-c2a5-4e8f-b59a-9014b35e1666 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:00.007266 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c60cb3-56a9-4fd5-bc47-0c2fec512e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:00.008085 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c60cb3-56a9-4fd5-bc47-0c2fec512e62 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:00.008374 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c60cb3-56a9-4fd5-bc47-0c2fec512e62 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:00.008653 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c60cb3-56a9-4fd5-bc47-0c2fec512e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:00.031349 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2c537-327e-4e3a-a833-81d37831d546 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:00.066698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e6cb8-fc02-4d0c-a0c6-863cec08b7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:00.067526 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e6cb8-fc02-4d0c-a0c6-863cec08b7e3 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:00.067811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e6cb8-fc02-4d0c-a0c6-863cec08b7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:00.068114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e6cb8-fc02-4d0c-a0c6-863cec08b7e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:00.069391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e6cb8-fc02-4d0c-a0c6-863cec08b7e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:00.070833 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e6cb8-fc02-4d0c-a0c6-863cec08b7e3 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:00.072538 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 10/18] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:11:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:00.079454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e6cb8-fc02-4d0c-a0c6-863cec08b7e3 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:00.135436 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c60cb3-56a9-4fd5-bc47-0c2fec512e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:00.136837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c60cb3-56a9-4fd5-bc47-0c2fec512e62 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-d9c60cb3-56a9-4fd5-bc47-0c2fec512e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:12:00.157933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:00.158891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:00.159917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:12:00.159917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:12:00.159917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:12:00.159917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:00.159917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 9/19] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:11:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:12:00.166146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c60cb3-56a9-4fd5-bc47-0c2fec512e62 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:00.179753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e6cb8-fc02-4d0c-a0c6-863cec08b7e3 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:00.211096 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcc84a-96f6-4804-b92a-3a1af8eb7d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:00.211745 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcc84a-96f6-4804-b92a-3a1af8eb7d5d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:00.212058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcc84a-96f6-4804-b92a-3a1af8eb7d5d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:00.212337 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcc84a-96f6-4804-b92a-3a1af8eb7d5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:00.213164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcc84a-96f6-4804-b92a-3a1af8eb7d5d None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.domain.id=default, user.id=c49606bc956845408ea8db21c53f217a) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:00.213475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcc84a-96f6-4804-b92a-3a1af8eb7d5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:00.214906 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcc84a-96f6-4804-b92a-3a1af8eb7d5d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:00.221803 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 11/20] 10.222.0.44 () {56 vars in 1322 bytes} [Mon May 9 04:12:00 2022] GET /identity/v3/role_assignments?user.id=c49606bc956845408ea8db21c53f217a&scope.domain.id=default&role.id=e59113c0366049e889df3f55eed90c72 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:00.264036 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c60cb3-56a9-4fd5-bc47-0c2fec512e62 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:00.299241 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2e399-2066-41ba-a385-a275780c05c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:00.299949 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2e399-2066-41ba-a385-a275780c05c1 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:00.300239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2e399-2066-41ba-a385-a275780c05c1 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:00.300515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2e399-2066-41ba-a385-a275780c05c1 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:00.301261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2e399-2066-41ba-a385-a275780c05c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:00.301572 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2e399-2066-41ba-a385-a275780c05c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:00.302990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2e399-2066-41ba-a385-a275780c05c1 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:00.322269 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 10/21] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:02.925062 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fe84f-1eaa-452d-831b-a071a1fed97e None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:02.926097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fe84f-1eaa-452d-831b-a071a1fed97e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:02.926608 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fe84f-1eaa-452d-831b-a071a1fed97e None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:02.928472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 12/22] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:02.941865 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195d965-7cb1-4e43-be54-78ee9779c73e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:02.942771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195d965-7cb1-4e43-be54-78ee9779c73e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:02.943275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195d965-7cb1-4e43-be54-78ee9779c73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:02.964687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-2195d965-7cb1-4e43-be54-78ee9779c73e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:02.966198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-2195d965-7cb1-4e43-be54-78ee9779c73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:03.004729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5881d9-c134-4dd5-a3ca-ea8db9718be5 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:03.005033 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5881d9-c134-4dd5-a3ca-ea8db9718be5 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:03.005296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5881d9-c134-4dd5-a3ca-ea8db9718be5 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:03.012117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 13/23] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:03.013261 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8b632-60b7-4139-a121-9823b1ae6e1a None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:03.013553 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8b632-60b7-4139-a121-9823b1ae6e1a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:03.013836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8b632-60b7-4139-a121-9823b1ae6e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:03.050258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e6c8b632-60b7-4139-a121-9823b1ae6e1a None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:03.051461 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e6c8b632-60b7-4139-a121-9823b1ae6e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:03.082341 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 11/24] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:03.118373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195d965-7cb1-4e43-be54-78ee9779c73e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:03.170487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 14/25] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:03.214544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8b632-60b7-4139-a121-9823b1ae6e1a None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:03.340644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8b632-60b7-4139-a121-9823b1ae6e1a None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:03.345976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195d965-7cb1-4e43-be54-78ee9779c73e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:03.383149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361bb42-8f98-4313-a79e-47d1fbcb021e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:03.384011 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361bb42-8f98-4313-a79e-47d1fbcb021e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:03.384301 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361bb42-8f98-4313-a79e-47d1fbcb021e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:03.384579 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361bb42-8f98-4313-a79e-47d1fbcb021e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:03.406276 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361bb42-8f98-4313-a79e-47d1fbcb021e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:03.424005 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1501fe-e987-4547-bdc5-35ee31512fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:03.424822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1501fe-e987-4547-bdc5-35ee31512fea None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:03.425112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1501fe-e987-4547-bdc5-35ee31512fea None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:03.425395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1501fe-e987-4547-bdc5-35ee31512fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-7361bb42-8f98-4313-a79e-47d1fbcb021e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:03.429614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:03.430422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:03.431165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:03.431165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:03.431165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:03.431165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:03.431165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:03.431165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:03.431165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:03.431165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:03.431165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:03.431165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:03.431165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:03.431165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:03.431165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 15/26] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:12:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:03.432819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd775a2e-b81b-4430-9a14-b0cce2578026 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:03.432819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd775a2e-b81b-4430-9a14-b0cce2578026 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:03.432819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd775a2e-b81b-4430-9a14-b0cce2578026 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:03.440662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 16/27] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:03.442114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd775a2e-b81b-4430-9a14-b0cce2578026 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:03.443811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1501fe-e987-4547-bdc5-35ee31512fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:03.445386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1501fe-e987-4547-bdc5-35ee31512fea None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-ae1501fe-e987-4547-bdc5-35ee31512fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:12:03.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:03.480870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:03.481643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:12:03.481643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:12:03.481643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:12:03.481643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:03.484632 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 12/28] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:03.486095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a516d4-da43-4818-bd26-896eb81e1948 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:03.486677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a516d4-da43-4818-bd26-896eb81e1948 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:03.487173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a516d4-da43-4818-bd26-896eb81e1948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:03.523164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-40a516d4-da43-4818-bd26-896eb81e1948 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:03.525606 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-40a516d4-da43-4818-bd26-896eb81e1948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:03.604004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd775a2e-b81b-4430-9a14-b0cce2578026 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:03.626088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 13/29] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:03.627331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a516d4-da43-4818-bd26-896eb81e1948 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:03.658496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae111f-51d9-4b7f-8fd0-fb21e5f2e6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:03.659176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae111f-51d9-4b7f-8fd0-fb21e5f2e6d5 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:03.659457 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae111f-51d9-4b7f-8fd0-fb21e5f2e6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:03.659730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae111f-51d9-4b7f-8fd0-fb21e5f2e6d5 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:03.660518 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae111f-51d9-4b7f-8fd0-fb21e5f2e6d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:03.660821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae111f-51d9-4b7f-8fd0-fb21e5f2e6d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:03.662219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae111f-51d9-4b7f-8fd0-fb21e5f2e6d5 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:03.676057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 17/30] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:03.677507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d35d4-836f-43fc-ad15-27ae50e7a0fe None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:03.677802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d35d4-836f-43fc-ad15-27ae50e7a0fe None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:03.678080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d35d4-836f-43fc-ad15-27ae50e7a0fe None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:03.683203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 18/31] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:03.683203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d35d4-836f-43fc-ad15-27ae50e7a0fe None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:03.753684 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a516d4-da43-4818-bd26-896eb81e1948 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:03.791739 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd537b2f-e4cb-4ee7-a07b-00f29caf8c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:03.792726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd537b2f-e4cb-4ee7-a07b-00f29caf8c97 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:03.793208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd537b2f-e4cb-4ee7-a07b-00f29caf8c97 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:03.793698 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd537b2f-e4cb-4ee7-a07b-00f29caf8c97 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:03.794659 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd537b2f-e4cb-4ee7-a07b-00f29caf8c97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:03.795155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd537b2f-e4cb-4ee7-a07b-00f29caf8c97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:03.796844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd537b2f-e4cb-4ee7-a07b-00f29caf8c97 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:03.809067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 14/32] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:03.810156 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd537b2f-e4cb-4ee7-a07b-00f29caf8c97 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:03.831948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d35d4-836f-43fc-ad15-27ae50e7a0fe None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:03.875292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b27bd0-3404-4d84-9b9e-ab53e90001eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:03.875912 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b27bd0-3404-4d84-9b9e-ab53e90001eb None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:03.876205 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b27bd0-3404-4d84-9b9e-ab53e90001eb None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:03.876484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b27bd0-3404-4d84-9b9e-ab53e90001eb None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:03.877397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b27bd0-3404-4d84-9b9e-ab53e90001eb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:03.879306 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b27bd0-3404-4d84-9b9e-ab53e90001eb None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:03.913492 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd537b2f-e4cb-4ee7-a07b-00f29caf8c97 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:03.951931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c7c5b-23b3-480a-a1a6-849d1a1bdb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:03.953795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c7c5b-23b3-480a-a1a6-849d1a1bdb67 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:03.954085 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c7c5b-23b3-480a-a1a6-849d1a1bdb67 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:03.954361 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c7c5b-23b3-480a-a1a6-849d1a1bdb67 None admin] PATH_INFO: `/v3/users/c49606bc956845408ea8db21c53f217a` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:03.955903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c7c5b-23b3-480a-a1a6-849d1a1bdb67 None admin] RBAC: Authorizing `identity:get_user(user_id=c49606bc956845408ea8db21c53f217a)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:03.957516 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c7c5b-23b3-480a-a1a6-849d1a1bdb67 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:03.963713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 15/33] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:03 2022] GET /identity/v3/users/c49606bc956845408ea8db21c53f217a => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:03.963713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fe88a-f611-475b-8e30-cb424841cc30 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:03.963713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fe88a-f611-475b-8e30-cb424841cc30 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:03.963713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fe88a-f611-475b-8e30-cb424841cc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:03.984007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-046fe88a-f611-475b-8e30-cb424841cc30 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:03.985665 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-046fe88a-f611-475b-8e30-cb424841cc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:04.028110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 19/34] 10.222.0.44 () {60 vars in 1131 bytes} [Mon May 9 04:12:03 2022] POST /identity/v3/roles => generated 225 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:12:04.029543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b27bd0-3404-4d84-9b9e-ab53e90001eb None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:04.061744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 16/35] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:04.063596 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fe88a-f611-475b-8e30-cb424841cc30 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:04.175468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fe88a-f611-475b-8e30-cb424841cc30 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:04.212773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe74b7-d122-447e-b2f8-25723ca9a185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:04.213965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe74b7-d122-447e-b2f8-25723ca9a185 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:04.214476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe74b7-d122-447e-b2f8-25723ca9a185 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:04.214954 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe74b7-d122-447e-b2f8-25723ca9a185 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:04.216538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe74b7-d122-447e-b2f8-25723ca9a185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:04.218874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe74b7-d122-447e-b2f8-25723ca9a185 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:04.220823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 17/36] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:04.221770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe74b7-d122-447e-b2f8-25723ca9a185 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:04.228240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b27bd0-3404-4d84-9b9e-ab53e90001eb None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:04.265090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e0582-a0a1-42cc-a117-37fde92d785f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:04.265861 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e0582-a0a1-42cc-a117-37fde92d785f None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:04.266155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e0582-a0a1-42cc-a117-37fde92d785f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:04.266437 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e0582-a0a1-42cc-a117-37fde92d785f None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:04.267510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e0582-a0a1-42cc-a117-37fde92d785f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:04.268786 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e0582-a0a1-42cc-a117-37fde92d785f None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:04.302767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 20/37] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:12:04 2022] POST /identity/v3/projects => generated 301 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:12:04.303733 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e0582-a0a1-42cc-a117-37fde92d785f None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:04.386137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe74b7-d122-447e-b2f8-25723ca9a185 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:04.441370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84117224-913e-4b10-b608-a9ea3c313bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:04.443547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84117224-913e-4b10-b608-a9ea3c313bf1 None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:04.443844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84117224-913e-4b10-b608-a9ea3c313bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:04.444128 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e0582-a0a1-42cc-a117-37fde92d785f None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:04.444225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84117224-913e-4b10-b608-a9ea3c313bf1 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:04.444929 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84117224-913e-4b10-b608-a9ea3c313bf1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:04.446669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84117224-913e-4b10-b608-a9ea3c313bf1 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:04.492113 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7659a-0846-41a9-ad8e-1a31f487c15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:04.493053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7659a-0846-41a9-ad8e-1a31f487c15a None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:04.493343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7659a-0846-41a9-ad8e-1a31f487c15a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:04.493632 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7659a-0846-41a9-ad8e-1a31f487c15a None admin] PATH_INFO: `/v3/domains/default/users/c49606bc956845408ea8db21c53f217a/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:04.495772 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7659a-0846-41a9-ad8e-1a31f487c15a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c49606bc956845408ea8db21c53f217a, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:04.497370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7659a-0846-41a9-ad8e-1a31f487c15a None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:04.518093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 21/38] 10.222.0.44 () {58 vars in 1361 bytes} [Mon May 9 04:12:04 2022] PUT /identity/v3/domains/default/users/c49606bc956845408ea8db21c53f217a/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:12:04.525259 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e078e19-a1a7-4116-9664-b9134c7d474d None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:04.525614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e078e19-a1a7-4116-9664-b9134c7d474d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:04.525891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e078e19-a1a7-4116-9664-b9134c7d474d None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:04.527289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 22/39] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:04.536121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e1db1-fd4b-4418-91c6-c188507710ef None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:04.536409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e1db1-fd4b-4418-91c6-c188507710ef None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:04.536695 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e1db1-fd4b-4418-91c6-c188507710ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:04.574985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-fb1e1db1-fd4b-4418-91c6-c188507710ef None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:04.576650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-fb1e1db1-fd4b-4418-91c6-c188507710ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:04.592374 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 18/40] 10.222.0.44 () {60 vars in 1131 bytes} [Mon May 9 04:12:04 2022] POST /identity/v3/roles => generated 219 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:12:04.644094 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d764282-ea47-4165-86ac-d08b2e5abdf2 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:04.644722 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d764282-ea47-4165-86ac-d08b2e5abdf2 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:04.645225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d764282-ea47-4165-86ac-d08b2e5abdf2 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:04.646806 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 19/41] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:12:04.654163 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377c2c-0092-4309-9ed9-013123ea87af None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:04.654452 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377c2c-0092-4309-9ed9-013123ea87af None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:04.654732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377c2c-0092-4309-9ed9-013123ea87af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:04.673626 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 23/42] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:04.691553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-98377c2c-0092-4309-9ed9-013123ea87af None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:04.693617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-98377c2c-0092-4309-9ed9-013123ea87af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:04.702083 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e1db1-fd4b-4418-91c6-c188507710ef None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:04.791232 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 20/43] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:04.817095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98377c2c-0092-4309-9ed9-013123ea87af None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:04.840664 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e1db1-fd4b-4418-91c6-c188507710ef None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:04.880755 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5801e-9089-4009-8e56-5b8285fc5752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:04.881436 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5801e-9089-4009-8e56-5b8285fc5752 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:04.881721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5801e-9089-4009-8e56-5b8285fc5752 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:04.881999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5801e-9089-4009-8e56-5b8285fc5752 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:04.882763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5801e-9089-4009-8e56-5b8285fc5752 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:04.884208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5801e-9089-4009-8e56-5b8285fc5752 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:04.903769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 24/44] 10.222.0.44 () {60 vars in 1131 bytes} [Mon May 9 04:12:04 2022] POST /identity/v3/roles => generated 223 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:12:04.933948 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98377c2c-0092-4309-9ed9-013123ea87af None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:04.972061 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19435c67-e85b-4f79-8387-d4fdaa5915de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:04.973946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19435c67-e85b-4f79-8387-d4fdaa5915de None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:04.974545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19435c67-e85b-4f79-8387-d4fdaa5915de None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:04.975067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19435c67-e85b-4f79-8387-d4fdaa5915de None admin] PATH_INFO: `/v3/domains/default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:04.976709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19435c67-e85b-4f79-8387-d4fdaa5915de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:04.978701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19435c67-e85b-4f79-8387-d4fdaa5915de None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:04.980935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 21/45] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:04.987438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5801e-9089-4009-8e56-5b8285fc5752 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:05.092846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5801e-9089-4009-8e56-5b8285fc5752 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:05.126903 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66574931-8e93-4855-ab9e-32fd3ea9a52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:05.127820 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66574931-8e93-4855-ab9e-32fd3ea9a52b None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:05.128408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66574931-8e93-4855-ab9e-32fd3ea9a52b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:05.128689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66574931-8e93-4855-ab9e-32fd3ea9a52b None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:05.129415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66574931-8e93-4855-ab9e-32fd3ea9a52b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:05.130884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66574931-8e93-4855-ab9e-32fd3ea9a52b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:05.157715 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 25/46] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:12:04 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:12:06.889904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a9aef-37e1-4e2c-821a-104df68af390 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:06.890897 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a9aef-37e1-4e2c-821a-104df68af390 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:06.891064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a9aef-37e1-4e2c-821a-104df68af390 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:06.892538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 22/47] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:06.897831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70469fb-77dc-4176-8813-6d1000529a14 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:06.898252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70469fb-77dc-4176-8813-6d1000529a14 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:06.898407 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70469fb-77dc-4176-8813-6d1000529a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:06.914493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e70469fb-77dc-4176-8813-6d1000529a14 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:06.915663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e70469fb-77dc-4176-8813-6d1000529a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:06.978637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 26/48] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:07.002097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a9aef-37e1-4e2c-821a-104df68af390 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:07.100064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a9aef-37e1-4e2c-821a-104df68af390 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:07.134583 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc1553-63df-449d-83ae-6d373d506b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:07.135390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc1553-63df-449d-83ae-6d373d506b9b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:07.135682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc1553-63df-449d-83ae-6d373d506b9b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:07.135994 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc1553-63df-449d-83ae-6d373d506b9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:07.147119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc1553-63df-449d-83ae-6d373d506b9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-50dc1553-63df-449d-83ae-6d373d506b9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:07.156134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:07.157026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:07.158016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:07.158016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:07.158016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:07.158016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:07.158016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:07.158016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:07.158016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:07.158016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:07.158016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:07.158016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:07.158016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:07.158016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:07.158016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 23/49] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:12:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:07.163560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70469fb-77dc-4176-8813-6d1000529a14 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:07.267530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70469fb-77dc-4176-8813-6d1000529a14 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:07.299630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4a445-9ab7-46e0-89eb-5d6b8a803406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:07.300334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4a445-9ab7-46e0-89eb-5d6b8a803406 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:07.300623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4a445-9ab7-46e0-89eb-5d6b8a803406 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:07.300905 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4a445-9ab7-46e0-89eb-5d6b8a803406 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:07.301662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4a445-9ab7-46e0-89eb-5d6b8a803406 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:07.301976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4a445-9ab7-46e0-89eb-5d6b8a803406 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:07.303368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4a445-9ab7-46e0-89eb-5d6b8a803406 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:07.311089 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 27/50] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:07.317257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc1553-63df-449d-83ae-6d373d506b9b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:07.382058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf097d-0a24-44ba-839f-dc8b02b1742b None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:07.382265 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf097d-0a24-44ba-839f-dc8b02b1742b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:07.382553 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf097d-0a24-44ba-839f-dc8b02b1742b None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:07.383994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 28/51] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:12:07.389017 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4da0e-ec02-4210-9afe-51101dd12eb0 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:07.389307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4da0e-ec02-4210-9afe-51101dd12eb0 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:07.389588 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4da0e-ec02-4210-9afe-51101dd12eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:07.405483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c3b4da0e-ec02-4210-9afe-51101dd12eb0 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:07.406909 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c3b4da0e-ec02-4210-9afe-51101dd12eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:07.413772 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc1553-63df-449d-83ae-6d373d506b9b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:07.449750 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142a6d7-28b9-4de8-801b-4f225a11a332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:07.450630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a6d7-28b9-4de8-801b-4f225a11a332 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:07.450913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a6d7-28b9-4de8-801b-4f225a11a332 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:07.451190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a6d7-28b9-4de8-801b-4f225a11a332 None admin] PATH_INFO: `/v3/users/c49606bc956845408ea8db21c53f217a` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:07.452406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142a6d7-28b9-4de8-801b-4f225a11a332 None admin] RBAC: Authorizing `identity:get_user(user_id=c49606bc956845408ea8db21c53f217a)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:07.454248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142a6d7-28b9-4de8-801b-4f225a11a332 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:07.455899 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 24/52] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:07 2022] GET /identity/v3/users/c49606bc956845408ea8db21c53f217a => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:07.461250 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142a6d7-28b9-4de8-801b-4f225a11a332 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:07.470957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 29/53] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:07.493994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4da0e-ec02-4210-9afe-51101dd12eb0 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:07.554571 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142a6d7-28b9-4de8-801b-4f225a11a332 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:07.586515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c1703-a0b5-4094-9961-2e54ee64d15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:07.587236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4da0e-ec02-4210-9afe-51101dd12eb0 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:07.587358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c1703-a0b5-4094-9961-2e54ee64d15b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:07.587726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c1703-a0b5-4094-9961-2e54ee64d15b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:07.588051 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c1703-a0b5-4094-9961-2e54ee64d15b None admin] PATH_INFO: `/v3/domains/default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:07.589315 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c1703-a0b5-4094-9961-2e54ee64d15b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:07.590766 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c1703-a0b5-4094-9961-2e54ee64d15b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:07.592500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 25/54] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:07.598291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c1703-a0b5-4094-9961-2e54ee64d15b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:07.619382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4eaf5-c7be-414f-a745-41129e29733a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:07.620208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4eaf5-c7be-414f-a745-41129e29733a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:07.620495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4eaf5-c7be-414f-a745-41129e29733a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:07.620771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4eaf5-c7be-414f-a745-41129e29733a None admin] PATH_INFO: `/v3/domains/default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:07.621933 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4eaf5-c7be-414f-a745-41129e29733a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:07.623399 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4eaf5-c7be-414f-a745-41129e29733a None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:07.625599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 30/55] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:07.631082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4eaf5-c7be-414f-a745-41129e29733a None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:07.690357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c1703-a0b5-4094-9961-2e54ee64d15b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:07.721843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde9163-3bb0-4b5e-b43a-091342e5aaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:07.722088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4eaf5-c7be-414f-a745-41129e29733a None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:07.722502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde9163-3bb0-4b5e-b43a-091342e5aaac None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:07.722791 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde9163-3bb0-4b5e-b43a-091342e5aaac None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:07.723067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde9163-3bb0-4b5e-b43a-091342e5aaac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:07.723922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde9163-3bb0-4b5e-b43a-091342e5aaac None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.domain.id=default, user.id=c49606bc956845408ea8db21c53f217a) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:07.724239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde9163-3bb0-4b5e-b43a-091342e5aaac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:07.725679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde9163-3bb0-4b5e-b43a-091342e5aaac None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:07.735591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 26/56] 10.222.0.44 () {56 vars in 1322 bytes} [Mon May 9 04:12:07 2022] GET /identity/v3/role_assignments?user.id=c49606bc956845408ea8db21c53f217a&scope.domain.id=default&role.id=e59113c0366049e889df3f55eed90c72 => generated 530 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:07.767196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fce80-7729-473f-be67-d4b80718a54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:07.767887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fce80-7729-473f-be67-d4b80718a54b None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:07.768198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fce80-7729-473f-be67-d4b80718a54b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:07.768478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fce80-7729-473f-be67-d4b80718a54b None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:07.769232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fce80-7729-473f-be67-d4b80718a54b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:07.770743 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fce80-7729-473f-be67-d4b80718a54b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:07.802490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 31/57] 10.222.0.44 () {60 vars in 1140 bytes} [Mon May 9 04:12:07 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:12:10.051556 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60947d30-dac1-4f02-9f75-e363d83c41b9 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:10.052083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60947d30-dac1-4f02-9f75-e363d83c41b9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:10.052083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60947d30-dac1-4f02-9f75-e363d83c41b9 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:10.053437 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 27/58] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:10.058566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304eead-dff2-4352-a786-c031d5fb1ce0 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:10.058955 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304eead-dff2-4352-a786-c031d5fb1ce0 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:10.059127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304eead-dff2-4352-a786-c031d5fb1ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:10.075705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-3304eead-dff2-4352-a786-c031d5fb1ce0 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:10.077173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3304eead-dff2-4352-a786-c031d5fb1ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:10.142143 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 32/59] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:10.165988 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60947d30-dac1-4f02-9f75-e363d83c41b9 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:10.264497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60947d30-dac1-4f02-9f75-e363d83c41b9 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:10.296570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ece57-57dc-43ba-9d3c-4eab192d7b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:10.297379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ece57-57dc-43ba-9d3c-4eab192d7b78 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:10.297663 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ece57-57dc-43ba-9d3c-4eab192d7b78 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:10.297935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ece57-57dc-43ba-9d3c-4eab192d7b78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:10.299109 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ece57-57dc-43ba-9d3c-4eab192d7b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:10.300576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ece57-57dc-43ba-9d3c-4eab192d7b78 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:10.302234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 28/60] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:10.307853 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304eead-dff2-4352-a786-c031d5fb1ce0 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:10.400841 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304eead-dff2-4352-a786-c031d5fb1ce0 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:10.432049 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b02644-cac8-4ca9-82d4-7f65b89bc134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:10.432713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b02644-cac8-4ca9-82d4-7f65b89bc134 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:10.432999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b02644-cac8-4ca9-82d4-7f65b89bc134 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:10.433278 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b02644-cac8-4ca9-82d4-7f65b89bc134 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:10.433979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b02644-cac8-4ca9-82d4-7f65b89bc134 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:10.435411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b02644-cac8-4ca9-82d4-7f65b89bc134 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:10.461606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 33/61] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:12:10 2022] POST /identity/v3/users => generated 275 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:12:13.911945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dec4d-6c7b-4175-88b0-e71bacd89b56 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:13.911945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dec4d-6c7b-4175-88b0-e71bacd89b56 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:13.911945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dec4d-6c7b-4175-88b0-e71bacd89b56 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:13.912666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 29/62] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:13.920635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe4dcb-03b5-456b-a4b2-fcefb45bab15 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:13.920635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe4dcb-03b5-456b-a4b2-fcefb45bab15 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:13.920635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe4dcb-03b5-456b-a4b2-fcefb45bab15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:13.943113 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-49fe4dcb-03b5-456b-a4b2-fcefb45bab15 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:13.944635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-49fe4dcb-03b5-456b-a4b2-fcefb45bab15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:14.003021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17b7f8-84ab-48df-a3e1-93fa29355a4e None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.003642 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17b7f8-84ab-48df-a3e1-93fa29355a4e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.004148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17b7f8-84ab-48df-a3e1-93fa29355a4e None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.005740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 30/63] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:12:14.010862 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6948d1e-e44c-4284-9c81-1d10dfb0ab53 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.011381 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6948d1e-e44c-4284-9c81-1d10dfb0ab53 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.011876 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6948d1e-e44c-4284-9c81-1d10dfb0ab53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.026226 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 34/64] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:14.037502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-b6948d1e-e44c-4284-9c81-1d10dfb0ab53 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:14.038339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646ff71-3ff5-44e4-87a6-8af0bd400e01 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.038635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646ff71-3ff5-44e4-87a6-8af0bd400e01 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.038917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646ff71-3ff5-44e4-87a6-8af0bd400e01 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.039612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b6948d1e-e44c-4284-9c81-1d10dfb0ab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:14.040485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 35/65] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:12:14.045302 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907bdf9-8d09-43ee-8d44-fa9c09a530ec None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.045788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907bdf9-8d09-43ee-8d44-fa9c09a530ec None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.046747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907bdf9-8d09-43ee-8d44-fa9c09a530ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.063945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e907bdf9-8d09-43ee-8d44-fa9c09a530ec None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:14.065366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e907bdf9-8d09-43ee-8d44-fa9c09a530ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:14.142966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 31/66] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:14.144215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6948d1e-e44c-4284-9c81-1d10dfb0ab53 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:14.146215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 36/67] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:14.168944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907bdf9-8d09-43ee-8d44-fa9c09a530ec None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:14.266045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6948d1e-e44c-4284-9c81-1d10dfb0ab53 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:14.268451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907bdf9-8d09-43ee-8d44-fa9c09a530ec None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:14.302904 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41aa6a-3af2-45a1-9532-72d8f83e1e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:14.303249 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614d03e-fa5a-4b2a-8b68-61dab30f3509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:14.303723 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41aa6a-3af2-45a1-9532-72d8f83e1e43 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.304049 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41aa6a-3af2-45a1-9532-72d8f83e1e43 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.304142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614d03e-fa5a-4b2a-8b68-61dab30f3509 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.304329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41aa6a-3af2-45a1-9532-72d8f83e1e43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.304419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614d03e-fa5a-4b2a-8b68-61dab30f3509 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.304605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614d03e-fa5a-4b2a-8b68-61dab30f3509 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.314951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614d03e-fa5a-4b2a-8b68-61dab30f3509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:14.315257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41aa6a-3af2-45a1-9532-72d8f83e1e43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-3614d03e-fa5a-4b2a-8b68-61dab30f3509 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:14.322563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:14.323388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:14.324188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:14.324188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:14.324188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:14.324188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:14.324188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:14.324188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:14.324188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:14.324188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:14.324188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:14.324188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:14.324188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:14.324188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-af41aa6a-3af2-45a1-9532-72d8f83e1e43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:14.324759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:14.325510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 37/68] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:12:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:12:14.327354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41aa6a-3af2-45a1-9532-72d8f83e1e43 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:14.331153 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 32/69] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:14.331153 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0b0d6-8d56-4048-8f31-9398a340f783 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.331412 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0b0d6-8d56-4048-8f31-9398a340f783 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.331546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0b0d6-8d56-4048-8f31-9398a340f783 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.336351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 33/70] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:14.336351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0b0d6-8d56-4048-8f31-9398a340f783 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:14.429789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41aa6a-3af2-45a1-9532-72d8f83e1e43 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:14.434235 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0b0d6-8d56-4048-8f31-9398a340f783 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:14.465188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4189cf-e3db-4bcb-8b9d-35f0a9637ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:14.466111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4189cf-e3db-4bcb-8b9d-35f0a9637ae8 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.466399 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4189cf-e3db-4bcb-8b9d-35f0a9637ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.466678 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4189cf-e3db-4bcb-8b9d-35f0a9637ae8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.471004 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b50598-8196-492d-aa86-f912ccdba1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:14.471699 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b50598-8196-492d-aa86-f912ccdba1ac None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.472024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b50598-8196-492d-aa86-f912ccdba1ac None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.472302 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b50598-8196-492d-aa86-f912ccdba1ac None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.473067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b50598-8196-492d-aa86-f912ccdba1ac None admin] RBAC: Adding query filter params (name=member) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:14.473376 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b50598-8196-492d-aa86-f912ccdba1ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:14.474785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b50598-8196-492d-aa86-f912ccdba1ac None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:14.480560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4189cf-e3db-4bcb-8b9d-35f0a9637ae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:14.492148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 34/71] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-7e4189cf-e3db-4bcb-8b9d-35f0a9637ae8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:14.492320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:14.493119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:14.493872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:14.493872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:14.493872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:14.493872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:14.493872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:14.493872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:14.493872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:14.493872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:14.493872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:14.493872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:14.493872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:14.493872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:14.494448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b50598-8196-492d-aa86-f912ccdba1ac None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:14.494514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 38/72] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:12:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:14.494514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2f644-4c89-4189-8af9-d2fc4c47ea08 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.494514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2f644-4c89-4189-8af9-d2fc4c47ea08 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.494514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2f644-4c89-4189-8af9-d2fc4c47ea08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.509916 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-51b2f644-4c89-4189-8af9-d2fc4c47ea08 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:14.511089 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-51b2f644-4c89-4189-8af9-d2fc4c47ea08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:14.575565 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 39/73] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:14.576198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2f644-4c89-4189-8af9-d2fc4c47ea08 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:14.604022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b50598-8196-492d-aa86-f912ccdba1ac None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:14.639656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8228587-5cc9-478d-b25c-15185fefc843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:14.641135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8228587-5cc9-478d-b25c-15185fefc843 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.641427 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8228587-5cc9-478d-b25c-15185fefc843 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.641708 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8228587-5cc9-478d-b25c-15185fefc843 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.642462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8228587-5cc9-478d-b25c-15185fefc843 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:14.642776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8228587-5cc9-478d-b25c-15185fefc843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:14.644199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8228587-5cc9-478d-b25c-15185fefc843 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:14.652425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 35/74] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:12:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:14.653119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8228587-5cc9-478d-b25c-15185fefc843 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:14.673434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2f644-4c89-4189-8af9-d2fc4c47ea08 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:14.705278 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d977a-5ac3-4f2e-be04-c35cced3ff36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:14.706157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d977a-5ac3-4f2e-be04-c35cced3ff36 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.706438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d977a-5ac3-4f2e-be04-c35cced3ff36 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.706713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d977a-5ac3-4f2e-be04-c35cced3ff36 None admin] PATH_INFO: `/v3/users/3248cf2854434b85b81170d82133b435` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.719742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d977a-5ac3-4f2e-be04-c35cced3ff36 None admin] RBAC: Authorizing `identity:get_user(user_id=3248cf2854434b85b81170d82133b435)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:14.721232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d977a-5ac3-4f2e-be04-c35cced3ff36 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:14.722901 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 40/75] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:14 2022] GET /identity/v3/users/3248cf2854434b85b81170d82133b435 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:14.724419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d977a-5ac3-4f2e-be04-c35cced3ff36 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:14.746761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8228587-5cc9-478d-b25c-15185fefc843 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:14.779879 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969a85f-1db3-4450-a22d-e084b9af6cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:14.781163 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969a85f-1db3-4450-a22d-e084b9af6cf6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.781552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969a85f-1db3-4450-a22d-e084b9af6cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.781843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969a85f-1db3-4450-a22d-e084b9af6cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.782587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969a85f-1db3-4450-a22d-e084b9af6cf6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:14.782897 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969a85f-1db3-4450-a22d-e084b9af6cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:14.784303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969a85f-1db3-4450-a22d-e084b9af6cf6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:14.791913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 36/76] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:14.793143 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3a5df-6663-476a-9387-bd5fa5f7e6f9 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.793450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3a5df-6663-476a-9387-bd5fa5f7e6f9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.793726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3a5df-6663-476a-9387-bd5fa5f7e6f9 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.794964 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 37/77] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:14.795670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3a5df-6663-476a-9387-bd5fa5f7e6f9 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:14.818421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d977a-5ac3-4f2e-be04-c35cced3ff36 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:14.852954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff2b95-3c63-4971-8e6e-ed6e629fad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:14.853677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff2b95-3c63-4971-8e6e-ed6e629fad99 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.853960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff2b95-3c63-4971-8e6e-ed6e629fad99 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.854235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff2b95-3c63-4971-8e6e-ed6e629fad99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.864284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff2b95-3c63-4971-8e6e-ed6e629fad99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-cdff2b95-3c63-4971-8e6e-ed6e629fad99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:14.872155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:14.873126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:14.873995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:14.873995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:14.873995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:14.873995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:14.873995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:14.873995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:14.873995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:14.873995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:14.873995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:14.873995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:14.873995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:14.873995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:14.873995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 41/78] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:14.874962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff2b95-3c63-4971-8e6e-ed6e629fad99 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:14.900080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3a5df-6663-476a-9387-bd5fa5f7e6f9 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:14.932044 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ab3ac-799c-4b6a-987a-6966d57f654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:14.932923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ab3ac-799c-4b6a-987a-6966d57f654b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:14.933214 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ab3ac-799c-4b6a-987a-6966d57f654b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:14.933495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ab3ac-799c-4b6a-987a-6966d57f654b None admin] PATH_INFO: `/v3/users/3248cf2854434b85b81170d82133b435` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:14.934688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ab3ac-799c-4b6a-987a-6966d57f654b None admin] RBAC: Authorizing `identity:get_user(user_id=3248cf2854434b85b81170d82133b435)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:14.936160 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ab3ac-799c-4b6a-987a-6966d57f654b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:14.938798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 38/79] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:14 2022] GET /identity/v3/users/3248cf2854434b85b81170d82133b435 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:14.939328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ab3ac-799c-4b6a-987a-6966d57f654b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:14.973180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff2b95-3c63-4971-8e6e-ed6e629fad99 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:15.004116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9cf4b-9e29-4b1a-a805-5685d84047ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:15.004756 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9cf4b-9e29-4b1a-a805-5685d84047ad None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:15.005041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9cf4b-9e29-4b1a-a805-5685d84047ad None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:15.005319 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9cf4b-9e29-4b1a-a805-5685d84047ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:15.034691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ab3ac-799c-4b6a-987a-6966d57f654b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:15.065807 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f6604-e646-4ba6-b541-ce69b559c063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:15.066731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f6604-e646-4ba6-b541-ce69b559c063 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:15.067011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f6604-e646-4ba6-b541-ce69b559c063 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:15.067254 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9cf4b-9e29-4b1a-a805-5685d84047ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:15.067344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f6604-e646-4ba6-b541-ce69b559c063 None admin] PATH_INFO: `/v3/users/c49606bc956845408ea8db21c53f217a` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:15.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f6604-e646-4ba6-b541-ce69b559c063 None admin] RBAC: Authorizing `identity:get_user(user_id=c49606bc956845408ea8db21c53f217a)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:15.068763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9cf4b-9e29-4b1a-a805-5685d84047ad None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:15.069884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f6604-e646-4ba6-b541-ce69b559c063 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:15.071526 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 39/80] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:14 2022] GET /identity/v3/users/c49606bc956845408ea8db21c53f217a => generated 247 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:15.072672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a712e6d-db55-41a7-936f-208ce9fbb5db None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:15.072954 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a712e6d-db55-41a7-936f-208ce9fbb5db None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:15.073231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a712e6d-db55-41a7-936f-208ce9fbb5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:15.089381 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-9a712e6d-db55-41a7-936f-208ce9fbb5db None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:15.090647 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9a712e6d-db55-41a7-936f-208ce9fbb5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:15.152187 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 40/81] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:15.152924 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a712e6d-db55-41a7-936f-208ce9fbb5db None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:15.169196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 42/82] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:15.169196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9cf4b-9e29-4b1a-a805-5685d84047ad None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:15.251121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a712e6d-db55-41a7-936f-208ce9fbb5db None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:15.256707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9cf4b-9e29-4b1a-a805-5685d84047ad None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:15.282303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ac0ba-3949-4906-971a-b9d87250ce38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:15.282970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ac0ba-3949-4906-971a-b9d87250ce38 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:15.283257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ac0ba-3949-4906-971a-b9d87250ce38 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:15.283536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ac0ba-3949-4906-971a-b9d87250ce38 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:15.284315 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ac0ba-3949-4906-971a-b9d87250ce38 None admin] RBAC: Adding query filter params (name=member) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:15.284625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ac0ba-3949-4906-971a-b9d87250ce38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:15.285995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ac0ba-3949-4906-971a-b9d87250ce38 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:15.287943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35168bb-c45f-4d93-9bab-b2010bb47973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:15.288578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35168bb-c45f-4d93-9bab-b2010bb47973 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:15.288858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35168bb-c45f-4d93-9bab-b2010bb47973 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:15.289132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35168bb-c45f-4d93-9bab-b2010bb47973 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:15.293550 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 41/83] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:15.294337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ac0ba-3949-4906-971a-b9d87250ce38 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:15.351951 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35168bb-c45f-4d93-9bab-b2010bb47973 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:15.353409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35168bb-c45f-4d93-9bab-b2010bb47973 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:15.385561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ac0ba-3949-4906-971a-b9d87250ce38 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:15.417137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31542587-2ee6-47bc-b8ed-6eb241d4991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:15.417708 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31542587-2ee6-47bc-b8ed-6eb241d4991e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:15.417993 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31542587-2ee6-47bc-b8ed-6eb241d4991e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:15.418270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31542587-2ee6-47bc-b8ed-6eb241d4991e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:15.443263 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 43/84] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:15.444121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35168bb-c45f-4d93-9bab-b2010bb47973 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:15.486924 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31542587-2ee6-47bc-b8ed-6eb241d4991e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:15.488406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31542587-2ee6-47bc-b8ed-6eb241d4991e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:15.535857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35168bb-c45f-4d93-9bab-b2010bb47973 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:15.567969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f25a4-d3e7-450f-aaea-2e2d4b9cff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:15.568831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f25a4-d3e7-450f-aaea-2e2d4b9cff3d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:15.569113 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f25a4-d3e7-450f-aaea-2e2d4b9cff3d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:15.569391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f25a4-d3e7-450f-aaea-2e2d4b9cff3d None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:15.570597 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f25a4-d3e7-450f-aaea-2e2d4b9cff3d None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:15.572055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f25a4-d3e7-450f-aaea-2e2d4b9cff3d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:15.573732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 44/85] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:15 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:15.574487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f25a4-d3e7-450f-aaea-2e2d4b9cff3d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:15.584166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 42/86] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:15.584166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31542587-2ee6-47bc-b8ed-6eb241d4991e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:15.667798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f25a4-d3e7-450f-aaea-2e2d4b9cff3d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:15.670716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31542587-2ee6-47bc-b8ed-6eb241d4991e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:15.699277 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5527f3-12b5-4e13-b9d0-2b563b48bebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:15.700102 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5527f3-12b5-4e13-b9d0-2b563b48bebb None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:15.700393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5527f3-12b5-4e13-b9d0-2b563b48bebb None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:15.700671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5527f3-12b5-4e13-b9d0-2b563b48bebb None admin] PATH_INFO: `/v3/domains/default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:15.701419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3fc0e-7e34-4534-b0a0-7ccea8bb4926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:15.701862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5527f3-12b5-4e13-b9d0-2b563b48bebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:15.702859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3fc0e-7e34-4534-b0a0-7ccea8bb4926 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:15.703405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3fc0e-7e34-4534-b0a0-7ccea8bb4926 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:15.703680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3fc0e-7e34-4534-b0a0-7ccea8bb4926 None admin] PATH_INFO: `/v3/users/3248cf2854434b85b81170d82133b435` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:15.704253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5527f3-12b5-4e13-b9d0-2b563b48bebb None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:15.705035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3fc0e-7e34-4534-b0a0-7ccea8bb4926 None admin] RBAC: Authorizing `identity:get_user(user_id=3248cf2854434b85b81170d82133b435)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:15.705900 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 45/87] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:15.706458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3fc0e-7e34-4534-b0a0-7ccea8bb4926 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:15.706659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5527f3-12b5-4e13-b9d0-2b563b48bebb None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:15.708220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 43/88] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:15 2022] GET /identity/v3/users/3248cf2854434b85b81170d82133b435 => generated 275 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:15.712865 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3fc0e-7e34-4534-b0a0-7ccea8bb4926 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:15.805652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3fc0e-7e34-4534-b0a0-7ccea8bb4926 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:15.806220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5527f3-12b5-4e13-b9d0-2b563b48bebb None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:15.837997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded33f54-1768-442a-ac34-f0f385a3e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:15.838652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded33f54-1768-442a-ac34-f0f385a3e881 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:15.838935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded33f54-1768-442a-ac34-f0f385a3e881 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:15.839213 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded33f54-1768-442a-ac34-f0f385a3e881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:15.839409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81eccc-82df-4d2c-8257-a4d0dd3dc578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:15.840066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded33f54-1768-442a-ac34-f0f385a3e881 None admin] RBAC: Adding query filter params (role.id=d8b94102152a46768d568f66794854f3, scope.project.id=37e07551c481458b8e025b20f23fd735, user.id=3248cf2854434b85b81170d82133b435) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:15.840300 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81eccc-82df-4d2c-8257-a4d0dd3dc578 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:15.840386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded33f54-1768-442a-ac34-f0f385a3e881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:15.840581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81eccc-82df-4d2c-8257-a4d0dd3dc578 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:15.840855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81eccc-82df-4d2c-8257-a4d0dd3dc578 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:15.841787 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded33f54-1768-442a-ac34-f0f385a3e881 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:15.842062 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81eccc-82df-4d2c-8257-a4d0dd3dc578 None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:15.843881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81eccc-82df-4d2c-8257-a4d0dd3dc578 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:15.845595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 46/89] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:15 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:15.846424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81eccc-82df-4d2c-8257-a4d0dd3dc578 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:15.847158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 44/90] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:15 2022] GET /identity/v3/role_assignments?user.id=3248cf2854434b85b81170d82133b435&scope.project.id=37e07551c481458b8e025b20f23fd735&role.id=d8b94102152a46768d568f66794854f3 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:15.860368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded33f54-1768-442a-ac34-f0f385a3e881 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:15.955036 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81eccc-82df-4d2c-8257-a4d0dd3dc578 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:15.955422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded33f54-1768-442a-ac34-f0f385a3e881 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:15.989731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc226d1-8047-4077-90a0-f5bb905ac4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:15.989940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92b3e0-8d70-4f2b-8d5c-955ea9efdf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:15.990519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc226d1-8047-4077-90a0-f5bb905ac4ee None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:15.990674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92b3e0-8d70-4f2b-8d5c-955ea9efdf71 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:15.990785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc226d1-8047-4077-90a0-f5bb905ac4ee None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:15.990987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92b3e0-8d70-4f2b-8d5c-955ea9efdf71 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:15.991086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc226d1-8047-4077-90a0-f5bb905ac4ee None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:15.991243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92b3e0-8d70-4f2b-8d5c-955ea9efdf71 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:15.991764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc226d1-8047-4077-90a0-f5bb905ac4ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:15.992429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92b3e0-8d70-4f2b-8d5c-955ea9efdf71 None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:15.993202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc226d1-8047-4077-90a0-f5bb905ac4ee None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:15.993875 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92b3e0-8d70-4f2b-8d5c-955ea9efdf71 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:15.995773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 47/91] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:15 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:15.996533 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92b3e0-8d70-4f2b-8d5c-955ea9efdf71 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:16.027763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 45/92] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:12:15 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:12:16.027763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc226d1-8047-4077-90a0-f5bb905ac4ee None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:16.113534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92b3e0-8d70-4f2b-8d5c-955ea9efdf71 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:16.145528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc226d1-8047-4077-90a0-f5bb905ac4ee None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:16.153047 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9226244-4f3d-4b23-bea6-d510015d3d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:16.153767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9226244-4f3d-4b23-bea6-d510015d3d20 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:16.154105 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9226244-4f3d-4b23-bea6-d510015d3d20 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:16.154385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9226244-4f3d-4b23-bea6-d510015d3d20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:16.185492 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa371b5c-1a03-4da8-9734-07e416b00cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:16.186463 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa371b5c-1a03-4da8-9734-07e416b00cd2 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:16.187466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa371b5c-1a03-4da8-9734-07e416b00cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:16.187884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa371b5c-1a03-4da8-9734-07e416b00cd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:16.189096 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa371b5c-1a03-4da8-9734-07e416b00cd2 None admin] RBAC: Adding query filter params (role.id=7600100dce614d9bb4d47b1e660b7bb2, scope.project.id=37e07551c481458b8e025b20f23fd735, user.id=3248cf2854434b85b81170d82133b435) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:16.189601 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa371b5c-1a03-4da8-9734-07e416b00cd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:16.191247 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa371b5c-1a03-4da8-9734-07e416b00cd2 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:16.197820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 46/93] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:16 2022] GET /identity/v3/role_assignments?user.id=3248cf2854434b85b81170d82133b435&scope.project.id=37e07551c481458b8e025b20f23fd735&role.id=7600100dce614d9bb4d47b1e660b7bb2 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:16.199018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa371b5c-1a03-4da8-9734-07e416b00cd2 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:16.229723 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9226244-4f3d-4b23-bea6-d510015d3d20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:16.231491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9226244-4f3d-4b23-bea6-d510015d3d20 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:16.308843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa371b5c-1a03-4da8-9734-07e416b00cd2 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:16.343732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 48/94] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:16.347295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205bc73-c616-40ff-b65e-f61ea7c62ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:16.351938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205bc73-c616-40ff-b65e-f61ea7c62ca6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:16.351938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205bc73-c616-40ff-b65e-f61ea7c62ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:16.351938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205bc73-c616-40ff-b65e-f61ea7c62ca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:16.351938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205bc73-c616-40ff-b65e-f61ea7c62ca6 None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=37e07551c481458b8e025b20f23fd735, user.id=c49606bc956845408ea8db21c53f217a) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:16.351938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205bc73-c616-40ff-b65e-f61ea7c62ca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:16.351938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205bc73-c616-40ff-b65e-f61ea7c62ca6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:16.353726 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9226244-4f3d-4b23-bea6-d510015d3d20 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:16.357230 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 47/95] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:16 2022] GET /identity/v3/role_assignments?user.id=c49606bc956845408ea8db21c53f217a&scope.project.id=37e07551c481458b8e025b20f23fd735&role.id=e59113c0366049e889df3f55eed90c72 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:16.485371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9226244-4f3d-4b23-bea6-d510015d3d20 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:16.523511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c0454-c061-4bbf-9d94-6368a870507d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:16.524731 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c0454-c061-4bbf-9d94-6368a870507d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:16.525217 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c0454-c061-4bbf-9d94-6368a870507d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:16.525681 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c0454-c061-4bbf-9d94-6368a870507d None admin] PATH_INFO: `/v3/projects/a46b3ac7bb684582919f66ef0ee84b78` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:16.527172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c0454-c061-4bbf-9d94-6368a870507d None admin] RBAC: Authorizing `identity:get_project(project_id=a46b3ac7bb684582919f66ef0ee84b78)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:16.528999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c0454-c061-4bbf-9d94-6368a870507d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:16.531053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 49/96] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:16 2022] GET /identity/v3/projects/a46b3ac7bb684582919f66ef0ee84b78 => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:16.538120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205bc73-c616-40ff-b65e-f61ea7c62ca6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:16.657265 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205bc73-c616-40ff-b65e-f61ea7c62ca6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:16.693401 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48990ebf-322d-47a4-8fb9-2348127d6ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:16.694072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990ebf-322d-47a4-8fb9-2348127d6ef1 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:16.694360 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990ebf-322d-47a4-8fb9-2348127d6ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:16.694639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48990ebf-322d-47a4-8fb9-2348127d6ef1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:16.695472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48990ebf-322d-47a4-8fb9-2348127d6ef1 None admin] RBAC: Adding query filter params (role.id=d8b94102152a46768d568f66794854f3, scope.project.id=a46b3ac7bb684582919f66ef0ee84b78, user.id=3248cf2854434b85b81170d82133b435) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:16.695788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48990ebf-322d-47a4-8fb9-2348127d6ef1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:16.697334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48990ebf-322d-47a4-8fb9-2348127d6ef1 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:16.704037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 48/97] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:16 2022] GET /identity/v3/role_assignments?user.id=3248cf2854434b85b81170d82133b435&scope.project.id=a46b3ac7bb684582919f66ef0ee84b78&role.id=d8b94102152a46768d568f66794854f3 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:18.451510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4c78f-a845-4b0f-89cf-ab3607430267 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:18.453491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4c78f-a845-4b0f-89cf-ab3607430267 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:18.454000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4c78f-a845-4b0f-89cf-ab3607430267 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:18.455634 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 50/98] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:18.463926 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2199cde-ba5f-4d0c-b0cb-55e57446cbe8 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:18.464377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2199cde-ba5f-4d0c-b0cb-55e57446cbe8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:18.464570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2199cde-ba5f-4d0c-b0cb-55e57446cbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:18.482017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f2199cde-ba5f-4d0c-b0cb-55e57446cbe8 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:18.483207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f2199cde-ba5f-4d0c-b0cb-55e57446cbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:18.563433 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 49/99] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:18.598165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4c78f-a845-4b0f-89cf-ab3607430267 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:18.708945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4c78f-a845-4b0f-89cf-ab3607430267 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:18.764381 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1835b-8a7d-4d83-b1ac-07e18fec5258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:18.766114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1835b-8a7d-4d83-b1ac-07e18fec5258 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:18.766114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1835b-8a7d-4d83-b1ac-07e18fec5258 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:18.766435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1835b-8a7d-4d83-b1ac-07e18fec5258 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:18.785776 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1835b-8a7d-4d83-b1ac-07e18fec5258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-93c1835b-8a7d-4d83-b1ac-07e18fec5258 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:18.793689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:18.794578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:18.795382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:18.795382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:18.795382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:18.795382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:18.795382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:18.795382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:18.795382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:18.795382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:18.795382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:18.795382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:18.795382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:18.795382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:18.795382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 51/100] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:18.800996 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2199cde-ba5f-4d0c-b0cb-55e57446cbe8 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:18.917846 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2199cde-ba5f-4d0c-b0cb-55e57446cbe8 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:18.961977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb444c-a3da-4da3-8347-7ee72247a609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:18.962587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb444c-a3da-4da3-8347-7ee72247a609 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:18.962879 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb444c-a3da-4da3-8347-7ee72247a609 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:18.963165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb444c-a3da-4da3-8347-7ee72247a609 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:18.963946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb444c-a3da-4da3-8347-7ee72247a609 None admin] RBAC: Adding query filter params (name=member) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:18.964263 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb444c-a3da-4da3-8347-7ee72247a609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:18.965670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb444c-a3da-4da3-8347-7ee72247a609 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:18.978628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 50/101] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:18.984227 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1835b-8a7d-4d83-b1ac-07e18fec5258 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:19.104328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1835b-8a7d-4d83-b1ac-07e18fec5258 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:19.150784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042086d-3f6f-4e99-9403-3edc1b19f4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:19.151670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042086d-3f6f-4e99-9403-3edc1b19f4e2 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:19.151983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042086d-3f6f-4e99-9403-3edc1b19f4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:19.152261 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042086d-3f6f-4e99-9403-3edc1b19f4e2 None admin] PATH_INFO: `/v3/users/3248cf2854434b85b81170d82133b435` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:19.156655 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042086d-3f6f-4e99-9403-3edc1b19f4e2 None admin] RBAC: Authorizing `identity:get_user(user_id=3248cf2854434b85b81170d82133b435)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:19.158107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042086d-3f6f-4e99-9403-3edc1b19f4e2 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:19.160106 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 52/102] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:18 2022] GET /identity/v3/users/3248cf2854434b85b81170d82133b435 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:19.165138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb444c-a3da-4da3-8347-7ee72247a609 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:19.319635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb444c-a3da-4da3-8347-7ee72247a609 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:19.364599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347ad33-1a7e-48fe-a404-7504d93909e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:19.365486 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347ad33-1a7e-48fe-a404-7504d93909e9 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:19.365766 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347ad33-1a7e-48fe-a404-7504d93909e9 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:19.366038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347ad33-1a7e-48fe-a404-7504d93909e9 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:19.367235 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347ad33-1a7e-48fe-a404-7504d93909e9 None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:19.368694 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347ad33-1a7e-48fe-a404-7504d93909e9 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:19.370379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 51/103] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:19 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:19.376222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042086d-3f6f-4e99-9403-3edc1b19f4e2 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:19.522384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042086d-3f6f-4e99-9403-3edc1b19f4e2 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:19.524250 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5933e6-04e0-46dd-946c-ae6d30de6715 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:19.524640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5933e6-04e0-46dd-946c-ae6d30de6715 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:19.524822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5933e6-04e0-46dd-946c-ae6d30de6715 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:19.526172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 52/104] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:12:19.545384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b610f6-c17a-454c-9edb-353f01679684 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:19.545677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b610f6-c17a-454c-9edb-353f01679684 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:19.546018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b610f6-c17a-454c-9edb-353f01679684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:19.563043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629511f7-7148-430f-afb1-9077cbf16934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:19.563981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629511f7-7148-430f-afb1-9077cbf16934 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:19.564271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629511f7-7148-430f-afb1-9077cbf16934 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:19.564555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629511f7-7148-430f-afb1-9077cbf16934 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735/users/3248cf2854434b85b81170d82133b435/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:19.566547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629511f7-7148-430f-afb1-9077cbf16934 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e07551c481458b8e025b20f23fd735, user_id=3248cf2854434b85b81170d82133b435, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:19.568031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629511f7-7148-430f-afb1-9077cbf16934 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:19.576275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f5b610f6-c17a-454c-9edb-353f01679684 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:19.578604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f5b610f6-c17a-454c-9edb-353f01679684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:19.589207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 53/105] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:12:19 2022] PUT /identity/v3/projects/37e07551c481458b8e025b20f23fd735/users/3248cf2854434b85b81170d82133b435/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:12:19.590819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab53c92-d7dc-4a70-a3ae-0b3cff40b04f None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:19.591118 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab53c92-d7dc-4a70-a3ae-0b3cff40b04f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:19.591400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab53c92-d7dc-4a70-a3ae-0b3cff40b04f None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:19.592900 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 54/106] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:19.600421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c1658-46fe-412e-92f9-a79df80fd2dd None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:19.600771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c1658-46fe-412e-92f9-a79df80fd2dd None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:19.601061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c1658-46fe-412e-92f9-a79df80fd2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:19.619661 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-eb7c1658-46fe-412e-92f9-a79df80fd2dd None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:19.620970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-eb7c1658-46fe-412e-92f9-a79df80fd2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:19.715798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 55/107] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:19.717636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a7bdd-e560-43bb-9feb-de45f8e40a23 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:19.718468 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a7bdd-e560-43bb-9feb-de45f8e40a23 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:19.719188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a7bdd-e560-43bb-9feb-de45f8e40a23 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:19.720528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 56/108] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:19.726739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04c037-5685-416b-aaa3-ce319ed43238 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:19.727036 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04c037-5685-416b-aaa3-ce319ed43238 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:19.727324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04c037-5685-416b-aaa3-ce319ed43238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:19.750464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-7e04c037-5685-416b-aaa3-ce319ed43238 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:19.751698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-7e04c037-5685-416b-aaa3-ce319ed43238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:19.767196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 53/109] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:19.768252 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b610f6-c17a-454c-9edb-353f01679684 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:19.840209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 57/110] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:19.841047 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04c037-5685-416b-aaa3-ce319ed43238 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:19.930354 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b610f6-c17a-454c-9edb-353f01679684 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:19.967121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6553c-e8fd-43b1-b6c5-bea3435c29e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:19.968961 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6553c-e8fd-43b1-b6c5-bea3435c29e6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:19.970425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6553c-e8fd-43b1-b6c5-bea3435c29e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:19.970872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6553c-e8fd-43b1-b6c5-bea3435c29e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:19.971119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04c037-5685-416b-aaa3-ce319ed43238 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:19.982097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6553c-e8fd-43b1-b6c5-bea3435c29e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-31f6553c-e8fd-43b1-b6c5-bea3435c29e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:19.990709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:19.991746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:19.992595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:19.992595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:19.992595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:19.992595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:19.992595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:19.992595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:19.992595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:19.992595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:19.992595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:19.992595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:19.992595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:19.992595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:19.994548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 54/111] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:12:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:19.995543 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6553c-e8fd-43b1-b6c5-bea3435c29e6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:20.009531 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2e1ab-794e-48bf-95ef-2f214aab108e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:20.010637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2e1ab-794e-48bf-95ef-2f214aab108e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.010930 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2e1ab-794e-48bf-95ef-2f214aab108e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.011208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2e1ab-794e-48bf-95ef-2f214aab108e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.025153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2e1ab-794e-48bf-95ef-2f214aab108e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-d2b2e1ab-794e-48bf-95ef-2f214aab108e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:20.036154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:20.037059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:20.038008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:20.038008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:20.038008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:20.038008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:20.038008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:20.038008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:20.038008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:20.038008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:20.038008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:20.038008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:20.038008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:20.038008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:20.038008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 58/112] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:12:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:20.038658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2e1ab-794e-48bf-95ef-2f214aab108e None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:20.102066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6553c-e8fd-43b1-b6c5-bea3435c29e6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:20.141607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a1dd0-5911-4092-adce-22c922365e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:20.142768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a1dd0-5911-4092-adce-22c922365e39 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.143241 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a1dd0-5911-4092-adce-22c922365e39 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.143697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a1dd0-5911-4092-adce-22c922365e39 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.154992 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a1dd0-5911-4092-adce-22c922365e39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-e34a1dd0-5911-4092-adce-22c922365e39 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:20.163099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:20.164161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:20.164914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:20.164914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:20.164914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:20.164914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:20.164914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:20.164914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:20.164914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:20.164914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:20.164914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:20.164914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:20.164914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:20.164914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:20.167111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 55/113] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:20.172475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2e1ab-794e-48bf-95ef-2f214aab108e None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:20.173086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a1dd0-5911-4092-adce-22c922365e39 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:20.223460 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca5d4f-a836-4a0c-a0b2-218a5b89693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:20.224521 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca5d4f-a836-4a0c-a0b2-218a5b89693a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.224993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca5d4f-a836-4a0c-a0b2-218a5b89693a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.225607 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca5d4f-a836-4a0c-a0b2-218a5b89693a None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.227835 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca5d4f-a836-4a0c-a0b2-218a5b89693a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:20.228361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca5d4f-a836-4a0c-a0b2-218a5b89693a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:20.230663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca5d4f-a836-4a0c-a0b2-218a5b89693a None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:20.247496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 59/114] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:12:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:20.249274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca5d4f-a836-4a0c-a0b2-218a5b89693a None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:20.331476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a1dd0-5911-4092-adce-22c922365e39 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:20.371391 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66db12-f55f-43d3-a6ab-0bce6de14f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:20.372508 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66db12-f55f-43d3-a6ab-0bce6de14f36 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.372999 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66db12-f55f-43d3-a6ab-0bce6de14f36 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.373491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66db12-f55f-43d3-a6ab-0bce6de14f36 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.374505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66db12-f55f-43d3-a6ab-0bce6de14f36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:20.375013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66db12-f55f-43d3-a6ab-0bce6de14f36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:20.376630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66db12-f55f-43d3-a6ab-0bce6de14f36 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:20.384894 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 56/115] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:20.386022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66db12-f55f-43d3-a6ab-0bce6de14f36 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:20.398271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca5d4f-a836-4a0c-a0b2-218a5b89693a None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:20.441104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812ade6-dd92-4ede-bede-4dc2a30c3a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:20.441788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812ade6-dd92-4ede-bede-4dc2a30c3a0e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.442208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812ade6-dd92-4ede-bede-4dc2a30c3a0e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.442624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812ade6-dd92-4ede-bede-4dc2a30c3a0e None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.443396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812ade6-dd92-4ede-bede-4dc2a30c3a0e None admin] RBAC: Adding query filter params (name=reader) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:20.443707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812ade6-dd92-4ede-bede-4dc2a30c3a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:20.445637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812ade6-dd92-4ede-bede-4dc2a30c3a0e None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:20.458097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 60/116] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:20.463472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db6f9a-63f8-46fe-9fc9-a66a5625813f None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.464131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db6f9a-63f8-46fe-9fc9-a66a5625813f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.464761 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db6f9a-63f8-46fe-9fc9-a66a5625813f None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.468548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 61/117] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:20.469838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db6f9a-63f8-46fe-9fc9-a66a5625813f None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:20.499959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66db12-f55f-43d3-a6ab-0bce6de14f36 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:20.535267 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616775b-2af7-4009-ac2d-1c60c1b8aa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:20.536448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616775b-2af7-4009-ac2d-1c60c1b8aa28 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.536751 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616775b-2af7-4009-ac2d-1c60c1b8aa28 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.537020 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616775b-2af7-4009-ac2d-1c60c1b8aa28 None admin] PATH_INFO: `/v3/users/3248cf2854434b85b81170d82133b435` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.538245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616775b-2af7-4009-ac2d-1c60c1b8aa28 None admin] RBAC: Authorizing `identity:get_user(user_id=3248cf2854434b85b81170d82133b435)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:20.539719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616775b-2af7-4009-ac2d-1c60c1b8aa28 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:20.541459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 57/118] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:20 2022] GET /identity/v3/users/3248cf2854434b85b81170d82133b435 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:20.542272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616775b-2af7-4009-ac2d-1c60c1b8aa28 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:20.581188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db6f9a-63f8-46fe-9fc9-a66a5625813f None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:20.615527 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ac10b-8883-409b-ad59-395221f49ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:20.616809 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ac10b-8883-409b-ad59-395221f49ee3 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.617326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ac10b-8883-409b-ad59-395221f49ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.617874 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ac10b-8883-409b-ad59-395221f49ee3 None admin] PATH_INFO: `/v3/users/c49606bc956845408ea8db21c53f217a` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.619267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ac10b-8883-409b-ad59-395221f49ee3 None admin] RBAC: Authorizing `identity:get_user(user_id=c49606bc956845408ea8db21c53f217a)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:20.621041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ac10b-8883-409b-ad59-395221f49ee3 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:20.623297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 62/119] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:20 2022] GET /identity/v3/users/c49606bc956845408ea8db21c53f217a => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:20.625403 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5a65b-4b76-4d83-9e26-780805e57295 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.626074 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5a65b-4b76-4d83-9e26-780805e57295 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.626554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5a65b-4b76-4d83-9e26-780805e57295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.643666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-acf5a65b-4b76-4d83-9e26-780805e57295 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:20.645304 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-acf5a65b-4b76-4d83-9e26-780805e57295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:20.647289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616775b-2af7-4009-ac2d-1c60c1b8aa28 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:20.680331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b5623-5d09-4524-94e4-b1748727b442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:20.681212 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b5623-5d09-4524-94e4-b1748727b442 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.681502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b5623-5d09-4524-94e4-b1748727b442 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.681782 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b5623-5d09-4524-94e4-b1748727b442 None admin] PATH_INFO: `/v3/users/666162c9a04948b295bb104f9f912b00` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.697229 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b5623-5d09-4524-94e4-b1748727b442 None admin] RBAC: Authorizing `identity:get_user(user_id=666162c9a04948b295bb104f9f912b00)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:20.698683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b5623-5d09-4524-94e4-b1748727b442 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:20.700401 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 58/120] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:20 2022] GET /identity/v3/users/666162c9a04948b295bb104f9f912b00 => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:20.701174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b5623-5d09-4524-94e4-b1748727b442 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:20.714559 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 63/121] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:20.715328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5a65b-4b76-4d83-9e26-780805e57295 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:20.804793 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b5623-5d09-4524-94e4-b1748727b442 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:20.818531 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5a65b-4b76-4d83-9e26-780805e57295 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:20.838145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0512453-35ee-477b-9a79-951f8119f065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:20.839011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0512453-35ee-477b-9a79-951f8119f065 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.839293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0512453-35ee-477b-9a79-951f8119f065 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.839568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0512453-35ee-477b-9a79-951f8119f065 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.840772 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0512453-35ee-477b-9a79-951f8119f065 None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:20.842194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0512453-35ee-477b-9a79-951f8119f065 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:20.848501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 59/122] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:20 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:20.848501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03118c46-9aa1-46a0-9774-a64550c81be4 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.848501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03118c46-9aa1-46a0-9774-a64550c81be4 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.848501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03118c46-9aa1-46a0-9774-a64550c81be4 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.848501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 60/123] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:20.848501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03118c46-9aa1-46a0-9774-a64550c81be4 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:20.857502 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b512f-15ae-463b-ac40-02e2805fdae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:20.860248 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b512f-15ae-463b-ac40-02e2805fdae8 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.860558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b512f-15ae-463b-ac40-02e2805fdae8 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.860848 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b512f-15ae-463b-ac40-02e2805fdae8 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.862043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b512f-15ae-463b-ac40-02e2805fdae8 None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:20.863486 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b512f-15ae-463b-ac40-02e2805fdae8 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:20.865206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 64/124] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:20 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:20.865960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b512f-15ae-463b-ac40-02e2805fdae8 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:20.950652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03118c46-9aa1-46a0-9774-a64550c81be4 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:20.962344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b512f-15ae-463b-ac40-02e2805fdae8 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:20.984480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105b5ee-d935-4928-915c-d05ce9b3fa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:20.985126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105b5ee-d935-4928-915c-d05ce9b3fa5b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.985409 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105b5ee-d935-4928-915c-d05ce9b3fa5b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.985695 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105b5ee-d935-4928-915c-d05ce9b3fa5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.995361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604198c7-e546-4367-b306-ddffa7a822f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:20.996171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604198c7-e546-4367-b306-ddffa7a822f0 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:20.996451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604198c7-e546-4367-b306-ddffa7a822f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:20.996727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604198c7-e546-4367-b306-ddffa7a822f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:20.997871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604198c7-e546-4367-b306-ddffa7a822f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:20.999825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604198c7-e546-4367-b306-ddffa7a822f0 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:21.001452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 65/125] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:21.002181 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604198c7-e546-4367-b306-ddffa7a822f0 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:21.053207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105b5ee-d935-4928-915c-d05ce9b3fa5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:21.054670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105b5ee-d935-4928-915c-d05ce9b3fa5b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:21.101608 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604198c7-e546-4367-b306-ddffa7a822f0 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:21.134989 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb534ab2-7d3e-4731-bc20-fa2445fff6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:21.135882 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb534ab2-7d3e-4731-bc20-fa2445fff6c6 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:21.136197 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb534ab2-7d3e-4731-bc20-fa2445fff6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:21.136466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb534ab2-7d3e-4731-bc20-fa2445fff6c6 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735/users/3248cf2854434b85b81170d82133b435/roles/7600100dce614d9bb4d47b1e660b7bb2` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:21.138636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb534ab2-7d3e-4731-bc20-fa2445fff6c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e07551c481458b8e025b20f23fd735, user_id=3248cf2854434b85b81170d82133b435, role_id=7600100dce614d9bb4d47b1e660b7bb2)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:21.140325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb534ab2-7d3e-4731-bc20-fa2445fff6c6 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:21.154663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 66/126] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:12:21 2022] PUT /identity/v3/projects/37e07551c481458b8e025b20f23fd735/users/3248cf2854434b85b81170d82133b435/roles/7600100dce614d9bb4d47b1e660b7bb2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:12:21.159883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 61/127] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:21.159883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f730a-0546-4508-ae16-13f2febc2763 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:21.159883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f730a-0546-4508-ae16-13f2febc2763 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:21.159883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f730a-0546-4508-ae16-13f2febc2763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:21.161480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb534ab2-7d3e-4731-bc20-fa2445fff6c6 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:21.177989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-6e5f730a-0546-4508-ae16-13f2febc2763 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:21.179293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-6e5f730a-0546-4508-ae16-13f2febc2763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:21.274680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 62/128] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:21.281024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f730a-0546-4508-ae16-13f2febc2763 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:21.281208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb534ab2-7d3e-4731-bc20-fa2445fff6c6 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:21.323610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf4e38-b74f-45f6-9d84-c02261e7a356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:21.325034 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf4e38-b74f-45f6-9d84-c02261e7a356 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:21.325331 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf4e38-b74f-45f6-9d84-c02261e7a356 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:21.325885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf4e38-b74f-45f6-9d84-c02261e7a356 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735/users/c49606bc956845408ea8db21c53f217a/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:21.329679 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf4e38-b74f-45f6-9d84-c02261e7a356 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e07551c481458b8e025b20f23fd735, user_id=c49606bc956845408ea8db21c53f217a, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:21.331290 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf4e38-b74f-45f6-9d84-c02261e7a356 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:21.351586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 67/129] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:12:21 2022] PUT /identity/v3/projects/37e07551c481458b8e025b20f23fd735/users/c49606bc956845408ea8db21c53f217a/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:12:21.353252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf4e38-b74f-45f6-9d84-c02261e7a356 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:21.420548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f730a-0546-4508-ae16-13f2febc2763 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:21.489070 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54872020-b0ba-4a69-9c72-723ef436f67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:21.489830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54872020-b0ba-4a69-9c72-723ef436f67f None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:21.490125 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54872020-b0ba-4a69-9c72-723ef436f67f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:21.490405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54872020-b0ba-4a69-9c72-723ef436f67f None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:21.491133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54872020-b0ba-4a69-9c72-723ef436f67f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:21.492640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54872020-b0ba-4a69-9c72-723ef436f67f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:21.505149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf4e38-b74f-45f6-9d84-c02261e7a356 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:21.564585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 63/130] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:12:21 2022] POST /identity/v3/projects => generated 302 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:12:21.565372 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54872020-b0ba-4a69-9c72-723ef436f67f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:21.569448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755b236-4a97-487e-b14d-dbed7b25f01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:21.570345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755b236-4a97-487e-b14d-dbed7b25f01b None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:21.570630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755b236-4a97-487e-b14d-dbed7b25f01b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:21.570909 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755b236-4a97-487e-b14d-dbed7b25f01b None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:21.573476 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755b236-4a97-487e-b14d-dbed7b25f01b None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:21.574977 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755b236-4a97-487e-b14d-dbed7b25f01b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:21.583569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 68/131] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:21 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:21.584575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755b236-4a97-487e-b14d-dbed7b25f01b None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:21.729070 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54872020-b0ba-4a69-9c72-723ef436f67f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:21.771964 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d6e1d-31c1-446b-bfdb-c56d13b6158e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:21.773261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d6e1d-31c1-446b-bfdb-c56d13b6158e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:21.773563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d6e1d-31c1-446b-bfdb-c56d13b6158e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:21.774127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d6e1d-31c1-446b-bfdb-c56d13b6158e None admin] PATH_INFO: `/v3/roles/member` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:21.782162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755b236-4a97-487e-b14d-dbed7b25f01b None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:21.793484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d6e1d-31c1-446b-bfdb-c56d13b6158e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-e40d6e1d-31c1-446b-bfdb-c56d13b6158e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:21.801971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:21.802824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:21.803587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:21.803587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:21.803587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:21.803587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:21.803587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:21.803587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:21.803587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:21.803587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:21.803587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:21.803587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:21.803587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:21.803587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:21.804198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 64/132] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:21.813558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d6e1d-31c1-446b-bfdb-c56d13b6158e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:21.847405 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc188a-b9bc-4547-824d-03d051bf9994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:21.848114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc188a-b9bc-4547-824d-03d051bf9994 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:21.848403 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc188a-b9bc-4547-824d-03d051bf9994 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:21.848681 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc188a-b9bc-4547-824d-03d051bf9994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:21.849530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc188a-b9bc-4547-824d-03d051bf9994 None admin] RBAC: Adding query filter params (role.id=2a69f0e80033468f96c13ef5f64d094d, scope.project.id=37e07551c481458b8e025b20f23fd735, user.id=666162c9a04948b295bb104f9f912b00) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:21.849843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc188a-b9bc-4547-824d-03d051bf9994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:21.851299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc188a-b9bc-4547-824d-03d051bf9994 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:21.869433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 69/133] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:21 2022] GET /identity/v3/role_assignments?user.id=666162c9a04948b295bb104f9f912b00&scope.project.id=37e07551c481458b8e025b20f23fd735&role.id=2a69f0e80033468f96c13ef5f64d094d => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:21.966729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d6e1d-31c1-446b-bfdb-c56d13b6158e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:22.016051 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28328d3d-0b2e-4413-b05e-fce3e76eb6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:22.016744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28328d3d-0b2e-4413-b05e-fce3e76eb6f9 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:22.017030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28328d3d-0b2e-4413-b05e-fce3e76eb6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:22.017309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28328d3d-0b2e-4413-b05e-fce3e76eb6f9 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:22.018066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28328d3d-0b2e-4413-b05e-fce3e76eb6f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:22.018373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28328d3d-0b2e-4413-b05e-fce3e76eb6f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:22.019762 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28328d3d-0b2e-4413-b05e-fce3e76eb6f9 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:22.028991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 65/134] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:22.034694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc188a-b9bc-4547-824d-03d051bf9994 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:22.147973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc188a-b9bc-4547-824d-03d051bf9994 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:22.196330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf7753-2ba7-4880-906b-67ec8676cf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:22.197636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf7753-2ba7-4880-906b-67ec8676cf96 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:22.198177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf7753-2ba7-4880-906b-67ec8676cf96 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:22.198658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf7753-2ba7-4880-906b-67ec8676cf96 None admin] PATH_INFO: `/v3/users/3248cf2854434b85b81170d82133b435` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:22.200367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf7753-2ba7-4880-906b-67ec8676cf96 None admin] RBAC: Authorizing `identity:get_user(user_id=3248cf2854434b85b81170d82133b435)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:22.202367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf7753-2ba7-4880-906b-67ec8676cf96 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:22.204732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 70/135] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:22 2022] GET /identity/v3/users/3248cf2854434b85b81170d82133b435 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:22.210547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28328d3d-0b2e-4413-b05e-fce3e76eb6f9 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:22.364672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28328d3d-0b2e-4413-b05e-fce3e76eb6f9 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:22.406330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ebf02-7636-4908-ad16-1fb7de229aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:22.407167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ebf02-7636-4908-ad16-1fb7de229aea None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:22.407451 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ebf02-7636-4908-ad16-1fb7de229aea None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:22.407733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ebf02-7636-4908-ad16-1fb7de229aea None admin] PATH_INFO: `/v3/projects/a46b3ac7bb684582919f66ef0ee84b78` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:22.408970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ebf02-7636-4908-ad16-1fb7de229aea None admin] RBAC: Authorizing `identity:get_project(project_id=a46b3ac7bb684582919f66ef0ee84b78)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:22.410419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ebf02-7636-4908-ad16-1fb7de229aea None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:22.412447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 66/136] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:22 2022] GET /identity/v3/projects/a46b3ac7bb684582919f66ef0ee84b78 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:22.417876 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf7753-2ba7-4880-906b-67ec8676cf96 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:22.492710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f423de-caf4-471c-8eb1-9eb0a10747b5 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:22.493021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f423de-caf4-471c-8eb1-9eb0a10747b5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:22.493595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f423de-caf4-471c-8eb1-9eb0a10747b5 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:22.494959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 67/137] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:22.500784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cc962-b739-4655-b4a2-aebe4d9191c3 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:22.501034 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cc962-b739-4655-b4a2-aebe4d9191c3 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:22.501315 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cc962-b739-4655-b4a2-aebe4d9191c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:22.532995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-940cc962-b739-4655-b4a2-aebe4d9191c3 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:22.534211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-940cc962-b739-4655-b4a2-aebe4d9191c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:22.562008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf7753-2ba7-4880-906b-67ec8676cf96 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:22.597914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b0b61-3bb9-44da-a8b8-a4de446a3838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:22.599653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b0b61-3bb9-44da-a8b8-a4de446a3838 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:22.600172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b0b61-3bb9-44da-a8b8-a4de446a3838 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:22.600664 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b0b61-3bb9-44da-a8b8-a4de446a3838 None admin] PATH_INFO: `/v3/projects/a46b3ac7bb684582919f66ef0ee84b78/users/3248cf2854434b85b81170d82133b435/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:22.603019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b0b61-3bb9-44da-a8b8-a4de446a3838 None admin] RBAC: Authorizing `identity:create_grant(project_id=a46b3ac7bb684582919f66ef0ee84b78, user_id=3248cf2854434b85b81170d82133b435, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:22.604722 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b0b61-3bb9-44da-a8b8-a4de446a3838 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:22.620969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 71/138] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:12:22 2022] PUT /identity/v3/projects/a46b3ac7bb684582919f66ef0ee84b78/users/3248cf2854434b85b81170d82133b435/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:12:22.631816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 68/139] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 09 04:12:22.655035 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b0b61-3bb9-44da-a8b8-a4de446a3838 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:22.769204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b0b61-3bb9-44da-a8b8-a4de446a3838 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:22.804169 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83638fb7-e127-4758-9e57-3118f2758e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:22.804983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83638fb7-e127-4758-9e57-3118f2758e13 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:22.805268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83638fb7-e127-4758-9e57-3118f2758e13 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:22.805547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83638fb7-e127-4758-9e57-3118f2758e13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:22.817034 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83638fb7-e127-4758-9e57-3118f2758e13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-83638fb7-e127-4758-9e57-3118f2758e13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:22.824652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:22.825549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:22.826935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:22.826935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:22.826935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:22.826935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:22.826935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:22.826935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:22.826935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:22.826935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:22.826935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:22.826935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:22.826935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:22.826935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:22.826935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 72/140] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:22.833034 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940cc962-b739-4655-b4a2-aebe4d9191c3 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:22.940201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940cc962-b739-4655-b4a2-aebe4d9191c3 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:22.976127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92eddf-a274-482a-93e3-94884208e31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:22.976806 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92eddf-a274-482a-93e3-94884208e31a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:22.977087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92eddf-a274-482a-93e3-94884208e31a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:22.977360 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92eddf-a274-482a-93e3-94884208e31a None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:22.978111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92eddf-a274-482a-93e3-94884208e31a None admin] RBAC: Adding query filter params (name=member) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:22.978414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92eddf-a274-482a-93e3-94884208e31a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:22.979790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92eddf-a274-482a-93e3-94884208e31a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:22.989454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 69/141] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:22.994726 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83638fb7-e127-4758-9e57-3118f2758e13 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:23.097367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83638fb7-e127-4758-9e57-3118f2758e13 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:23.129801 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be6479-f498-4b68-9bea-ee99ebeea21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:23.130690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6479-f498-4b68-9bea-ee99ebeea21a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:23.130973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6479-f498-4b68-9bea-ee99ebeea21a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:23.131250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be6479-f498-4b68-9bea-ee99ebeea21a None admin] PATH_INFO: `/v3/users/3248cf2854434b85b81170d82133b435` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:23.132488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be6479-f498-4b68-9bea-ee99ebeea21a None admin] RBAC: Authorizing `identity:get_user(user_id=3248cf2854434b85b81170d82133b435)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:23.133921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be6479-f498-4b68-9bea-ee99ebeea21a None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:23.135710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 73/142] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:22 2022] GET /identity/v3/users/3248cf2854434b85b81170d82133b435 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:23.141516 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92eddf-a274-482a-93e3-94884208e31a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:23.243157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92eddf-a274-482a-93e3-94884208e31a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:23.277124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b72059-9c3f-4725-9080-97a5c17fa09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:23.277779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b72059-9c3f-4725-9080-97a5c17fa09c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:23.278065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b72059-9c3f-4725-9080-97a5c17fa09c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:23.278339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b72059-9c3f-4725-9080-97a5c17fa09c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:23.344970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b72059-9c3f-4725-9080-97a5c17fa09c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:23.346521 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b72059-9c3f-4725-9080-97a5c17fa09c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:23.442677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 70/143] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:23.447993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be6479-f498-4b68-9bea-ee99ebeea21a None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:23.549167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be6479-f498-4b68-9bea-ee99ebeea21a None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:23.583097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5aa2f4-efa5-4415-87b8-1d786b001a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:23.584004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5aa2f4-efa5-4415-87b8-1d786b001a91 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:23.584292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5aa2f4-efa5-4415-87b8-1d786b001a91 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:23.584573 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5aa2f4-efa5-4415-87b8-1d786b001a91 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:23.585774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5aa2f4-efa5-4415-87b8-1d786b001a91 None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:23.587209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5aa2f4-efa5-4415-87b8-1d786b001a91 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:23.588922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 74/144] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:23 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:23.594850 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b72059-9c3f-4725-9080-97a5c17fa09c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:23.692671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b72059-9c3f-4725-9080-97a5c17fa09c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:23.726123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42289af-595e-4da6-b64a-a43d71e7664f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:23.726788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42289af-595e-4da6-b64a-a43d71e7664f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:23.727071 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42289af-595e-4da6-b64a-a43d71e7664f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:23.727344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42289af-595e-4da6-b64a-a43d71e7664f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:23.728200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42289af-595e-4da6-b64a-a43d71e7664f None admin] RBAC: Adding query filter params (role.id=d8b94102152a46768d568f66794854f3, scope.project.id=37e07551c481458b8e025b20f23fd735, user.id=3248cf2854434b85b81170d82133b435) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:23.728510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42289af-595e-4da6-b64a-a43d71e7664f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:23.729940 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42289af-595e-4da6-b64a-a43d71e7664f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:23.736715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 71/145] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:23 2022] GET /identity/v3/role_assignments?user.id=3248cf2854434b85b81170d82133b435&scope.project.id=37e07551c481458b8e025b20f23fd735&role.id=d8b94102152a46768d568f66794854f3 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:24.402886 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0eef0-e6a7-4e15-bc71-212c2ebc4fb8 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:24.403583 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0eef0-e6a7-4e15-bc71-212c2ebc4fb8 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:24.403583 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0eef0-e6a7-4e15-bc71-212c2ebc4fb8 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:24.404974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 75/146] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:24.411801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db267da6-10b7-44ea-9244-a95f56d1c16e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:24.412126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db267da6-10b7-44ea-9244-a95f56d1c16e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:24.412408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db267da6-10b7-44ea-9244-a95f56d1c16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:24.433124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-db267da6-10b7-44ea-9244-a95f56d1c16e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:24.434351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-db267da6-10b7-44ea-9244-a95f56d1c16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:24.518490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 72/147] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:24.555993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0eef0-e6a7-4e15-bc71-212c2ebc4fb8 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:24.679983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0eef0-e6a7-4e15-bc71-212c2ebc4fb8 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:24.713026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddaa618-937f-4b1b-a2f5-88cf01a95da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:24.713833 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaa618-937f-4b1b-a2f5-88cf01a95da5 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:24.714147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaa618-937f-4b1b-a2f5-88cf01a95da5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:24.714389 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaa618-937f-4b1b-a2f5-88cf01a95da5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:24.725033 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddaa618-937f-4b1b-a2f5-88cf01a95da5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-0ddaa618-937f-4b1b-a2f5-88cf01a95da5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:24.732992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:24.735103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:24.736956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:24.736956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:24.736956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:24.736956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:24.736956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:24.736956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:24.736956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:24.736956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:24.736956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:24.736956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:24.736956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:24.736956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:24.736956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 76/148] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:12:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:24.746225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db267da6-10b7-44ea-9244-a95f56d1c16e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:24.848497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db267da6-10b7-44ea-9244-a95f56d1c16e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:24.877395 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9c2de-98af-45fc-9a0c-38e37e1385d2 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:24.877687 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9c2de-98af-45fc-9a0c-38e37e1385d2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:24.877955 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9c2de-98af-45fc-9a0c-38e37e1385d2 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:24.879436 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 77/149] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:12:24.884651 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba09e65-f6b1-462f-9fbc-06539b3b6590 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:24.885162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba09e65-f6b1-462f-9fbc-06539b3b6590 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:24.885625 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba09e65-f6b1-462f-9fbc-06539b3b6590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:24.886876 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2429a7-d189-4a36-916a-640ea1246437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:24.887546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2429a7-d189-4a36-916a-640ea1246437 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:24.887830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2429a7-d189-4a36-916a-640ea1246437 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:24.888130 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2429a7-d189-4a36-916a-640ea1246437 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:24.895941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2429a7-d189-4a36-916a-640ea1246437 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:24.895941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2429a7-d189-4a36-916a-640ea1246437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:24.895941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2429a7-d189-4a36-916a-640ea1246437 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:24.900107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 73/150] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:12:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:24.908024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2429a7-d189-4a36-916a-640ea1246437 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:24.924962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-eba09e65-f6b1-462f-9fbc-06539b3b6590 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:24.926175 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-eba09e65-f6b1-462f-9fbc-06539b3b6590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:25.025896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2429a7-d189-4a36-916a-640ea1246437 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:25.064215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb0b65-8084-423f-b4e4-e4c2b1f2114b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:25.065146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb0b65-8084-423f-b4e4-e4c2b1f2114b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:25.065434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb0b65-8084-423f-b4e4-e4c2b1f2114b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:25.065710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb0b65-8084-423f-b4e4-e4c2b1f2114b None admin] PATH_INFO: `/v3/users/3248cf2854434b85b81170d82133b435` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:25.067269 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb0b65-8084-423f-b4e4-e4c2b1f2114b None admin] RBAC: Authorizing `identity:get_user(user_id=3248cf2854434b85b81170d82133b435)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:25.068927 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb0b65-8084-423f-b4e4-e4c2b1f2114b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:25.071531 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 74/151] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:24 2022] GET /identity/v3/users/3248cf2854434b85b81170d82133b435 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:25.081230 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb0b65-8084-423f-b4e4-e4c2b1f2114b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:25.085535 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 78/152] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:25.159253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba09e65-f6b1-462f-9fbc-06539b3b6590 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:25.229102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb0b65-8084-423f-b4e4-e4c2b1f2114b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:25.270122 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc795a-b374-4543-b432-97454630ab98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:25.270790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc795a-b374-4543-b432-97454630ab98 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:25.271082 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc795a-b374-4543-b432-97454630ab98 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:25.271365 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc795a-b374-4543-b432-97454630ab98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:25.272472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba09e65-f6b1-462f-9fbc-06539b3b6590 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:25.308804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076419ef-5d50-42fd-9afb-1d03e49796b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:25.309535 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076419ef-5d50-42fd-9afb-1d03e49796b3 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:25.309830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076419ef-5d50-42fd-9afb-1d03e49796b3 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:25.310106 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076419ef-5d50-42fd-9afb-1d03e49796b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:25.320906 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076419ef-5d50-42fd-9afb-1d03e49796b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-076419ef-5d50-42fd-9afb-1d03e49796b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:25.329445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:25.330296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:25.331075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:25.331075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:25.331075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:25.331075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:25.331075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:25.331075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:25.331075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:25.331075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:25.331075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:25.331075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:25.331075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:25.331075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:25.331075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 79/153] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:12:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:12:25.337167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ea2a8-cc2e-44bb-b1ba-1b3ad76e5169 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:25.337167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ea2a8-cc2e-44bb-b1ba-1b3ad76e5169 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:25.337167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ea2a8-cc2e-44bb-b1ba-1b3ad76e5169 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:25.337167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 80/154] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:25.338894 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ea2a8-cc2e-44bb-b1ba-1b3ad76e5169 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:25.363800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc795a-b374-4543-b432-97454630ab98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:25.365725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc795a-b374-4543-b432-97454630ab98 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:25.446793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ea2a8-cc2e-44bb-b1ba-1b3ad76e5169 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:25.472832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 75/155] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:25.474553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eadb46-bc88-47b8-a409-93a4471e3fa0 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:25.475086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eadb46-bc88-47b8-a409-93a4471e3fa0 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:25.475562 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eadb46-bc88-47b8-a409-93a4471e3fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:25.486453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e6c5b-0c5f-4b5b-9b07-cb153e862766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:25.487129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e6c5b-0c5f-4b5b-9b07-cb153e862766 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:25.487413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e6c5b-0c5f-4b5b-9b07-cb153e862766 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:25.487688 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e6c5b-0c5f-4b5b-9b07-cb153e862766 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:25.488477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e6c5b-0c5f-4b5b-9b07-cb153e862766 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:25.488787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e6c5b-0c5f-4b5b-9b07-cb153e862766 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:25.496380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-15eadb46-bc88-47b8-a409-93a4471e3fa0 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:25.496604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e6c5b-0c5f-4b5b-9b07-cb153e862766 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:25.498079 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-15eadb46-bc88-47b8-a409-93a4471e3fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:25.506463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 81/156] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:25.511859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e6c5b-0c5f-4b5b-9b07-cb153e862766 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:25.581529 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 76/157] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:25.582734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eadb46-bc88-47b8-a409-93a4471e3fa0 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:25.614871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e6c5b-0c5f-4b5b-9b07-cb153e862766 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:25.646545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47b2f1-d2c5-4aed-aa70-68e0b809b540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:25.647404 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47b2f1-d2c5-4aed-aa70-68e0b809b540 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:25.647681 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47b2f1-d2c5-4aed-aa70-68e0b809b540 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:25.647980 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47b2f1-d2c5-4aed-aa70-68e0b809b540 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:25.649147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47b2f1-d2c5-4aed-aa70-68e0b809b540 None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:25.650575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47b2f1-d2c5-4aed-aa70-68e0b809b540 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:25.652433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 82/158] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:25 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:25.653429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb53fdd-1da1-48df-bb3c-ee387035a5cb None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:25.653708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb53fdd-1da1-48df-bb3c-ee387035a5cb None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:25.653979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb53fdd-1da1-48df-bb3c-ee387035a5cb None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:25.655849 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 83/159] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:25.660486 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb53fdd-1da1-48df-bb3c-ee387035a5cb None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:25.685870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eadb46-bc88-47b8-a409-93a4471e3fa0 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:25.719005 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40077f-ca9f-47f6-bea8-cfa4b0418281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:25.719885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40077f-ca9f-47f6-bea8-cfa4b0418281 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:25.720195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40077f-ca9f-47f6-bea8-cfa4b0418281 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:25.720469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40077f-ca9f-47f6-bea8-cfa4b0418281 None admin] PATH_INFO: `/v3/users/c49606bc956845408ea8db21c53f217a` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:25.721651 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40077f-ca9f-47f6-bea8-cfa4b0418281 None admin] RBAC: Authorizing `identity:get_user(user_id=c49606bc956845408ea8db21c53f217a)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:25.723076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40077f-ca9f-47f6-bea8-cfa4b0418281 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:25.724731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 77/160] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:25 2022] GET /identity/v3/users/c49606bc956845408ea8db21c53f217a => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:25.725487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40077f-ca9f-47f6-bea8-cfa4b0418281 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:25.757274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb53fdd-1da1-48df-bb3c-ee387035a5cb None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:25.789050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5734b9-f34f-41d9-a028-0258be943989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:25.789834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5734b9-f34f-41d9-a028-0258be943989 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:25.790142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5734b9-f34f-41d9-a028-0258be943989 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:25.790410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5734b9-f34f-41d9-a028-0258be943989 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:25.791562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5734b9-f34f-41d9-a028-0258be943989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:25.793034 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5734b9-f34f-41d9-a028-0258be943989 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:25.794632 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 84/161] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:25.797479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020ef31-a247-4023-a516-d7e01b1846e0 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:25.797764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020ef31-a247-4023-a516-d7e01b1846e0 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:25.798038 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020ef31-a247-4023-a516-d7e01b1846e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:25.813466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-d020ef31-a247-4023-a516-d7e01b1846e0 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:25.814630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d020ef31-a247-4023-a516-d7e01b1846e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:25.823280 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40077f-ca9f-47f6-bea8-cfa4b0418281 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:25.855968 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a36567-802e-4e55-94ad-e0831d8dad7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:25.856547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a36567-802e-4e55-94ad-e0831d8dad7b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:25.856823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a36567-802e-4e55-94ad-e0831d8dad7b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:25.857094 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a36567-802e-4e55-94ad-e0831d8dad7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:25.857908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a36567-802e-4e55-94ad-e0831d8dad7b None admin] RBAC: Adding query filter params (role.id=7600100dce614d9bb4d47b1e660b7bb2, scope.project.id=37e07551c481458b8e025b20f23fd735, user.id=3248cf2854434b85b81170d82133b435) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:25.858213 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a36567-802e-4e55-94ad-e0831d8dad7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:25.859658 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a36567-802e-4e55-94ad-e0831d8dad7b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:25.865156 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 78/162] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:25 2022] GET /identity/v3/role_assignments?user.id=3248cf2854434b85b81170d82133b435&scope.project.id=37e07551c481458b8e025b20f23fd735&role.id=7600100dce614d9bb4d47b1e660b7bb2 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:25.866344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5a435-b0d4-4c00-ab2a-d335ca5d0759 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:25.866627 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5a435-b0d4-4c00-ab2a-d335ca5d0759 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:25.866901 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5a435-b0d4-4c00-ab2a-d335ca5d0759 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:25.868303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 79/163] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:25.868849 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5a435-b0d4-4c00-ab2a-d335ca5d0759 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:25.883282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 85/164] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:25.888391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020ef31-a247-4023-a516-d7e01b1846e0 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:25.987630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5a435-b0d4-4c00-ab2a-d335ca5d0759 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:25.994009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020ef31-a247-4023-a516-d7e01b1846e0 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:26.022254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02121937-15b2-4cb4-af90-98c77846277b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:26.022915 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02121937-15b2-4cb4-af90-98c77846277b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:26.023195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02121937-15b2-4cb4-af90-98c77846277b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:26.023469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02121937-15b2-4cb4-af90-98c77846277b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:26.030361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906b889-395f-46de-9b5a-8ce33665f2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:26.031116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906b889-395f-46de-9b5a-8ce33665f2d5 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:26.031421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906b889-395f-46de-9b5a-8ce33665f2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:26.031697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906b889-395f-46de-9b5a-8ce33665f2d5 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:26.032458 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906b889-395f-46de-9b5a-8ce33665f2d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:26.033937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906b889-395f-46de-9b5a-8ce33665f2d5 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:26.057731 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 86/165] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:12:25 2022] POST /identity/v3/users => generated 283 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:12:26.059387 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac723fb-6d85-4704-9f1c-570efe66e582 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:26.060794 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac723fb-6d85-4704-9f1c-570efe66e582 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:26.061308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac723fb-6d85-4704-9f1c-570efe66e582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:26.079589 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-cac723fb-6d85-4704-9f1c-570efe66e582 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:26.080841 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cac723fb-6d85-4704-9f1c-570efe66e582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:26.109560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02121937-15b2-4cb4-af90-98c77846277b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:26.110961 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02121937-15b2-4cb4-af90-98c77846277b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:26.176208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 87/166] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:26.177104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac723fb-6d85-4704-9f1c-570efe66e582 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:26.234383 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 80/167] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:26.235729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02121937-15b2-4cb4-af90-98c77846277b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:26.294569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac723fb-6d85-4704-9f1c-570efe66e582 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:26.336586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c3edc-cf66-4a3f-894b-6703d140f582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:26.337731 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c3edc-cf66-4a3f-894b-6703d140f582 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:26.338019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c3edc-cf66-4a3f-894b-6703d140f582 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:26.338356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c3edc-cf66-4a3f-894b-6703d140f582 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:26.348581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02121937-15b2-4cb4-af90-98c77846277b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:26.353262 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c3edc-cf66-4a3f-894b-6703d140f582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-116c3edc-cf66-4a3f-894b-6703d140f582 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:26.362511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:26.363586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 88/168] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:26.364545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c3edc-cf66-4a3f-894b-6703d140f582 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:26.391332 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5c060-085e-4815-80dd-d33df5f8df83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:26.392211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5c060-085e-4815-80dd-d33df5f8df83 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:26.392488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5c060-085e-4815-80dd-d33df5f8df83 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:26.392760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5c060-085e-4815-80dd-d33df5f8df83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:26.407066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5c060-085e-4815-80dd-d33df5f8df83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-d9f5c060-085e-4815-80dd-d33df5f8df83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:26.414852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:26.415990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 81/169] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:26.416946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5c060-085e-4815-80dd-d33df5f8df83 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:26.503937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c3edc-cf66-4a3f-894b-6703d140f582 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:26.535272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5c060-085e-4815-80dd-d33df5f8df83 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:26.551045 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527af08-4862-4ca7-9134-19fd6c28989c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:26.552058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527af08-4862-4ca7-9134-19fd6c28989c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:26.552343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527af08-4862-4ca7-9134-19fd6c28989c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:26.552619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527af08-4862-4ca7-9134-19fd6c28989c None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:26.553857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527af08-4862-4ca7-9134-19fd6c28989c None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:26.555383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527af08-4862-4ca7-9134-19fd6c28989c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:26.557299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 89/170] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:26 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:26.557910 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527af08-4862-4ca7-9134-19fd6c28989c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:26.573013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80adeb-cc2a-4fc1-b66a-9b854c7a58d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:26.574024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80adeb-cc2a-4fc1-b66a-9b854c7a58d2 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:26.574553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80adeb-cc2a-4fc1-b66a-9b854c7a58d2 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:26.574999 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80adeb-cc2a-4fc1-b66a-9b854c7a58d2 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:26.575947 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80adeb-cc2a-4fc1-b66a-9b854c7a58d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:26.576424 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80adeb-cc2a-4fc1-b66a-9b854c7a58d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:26.579044 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80adeb-cc2a-4fc1-b66a-9b854c7a58d2 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:26.589227 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 82/171] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:26.590072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80adeb-cc2a-4fc1-b66a-9b854c7a58d2 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:26.662989 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527af08-4862-4ca7-9134-19fd6c28989c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:26.697139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80adeb-cc2a-4fc1-b66a-9b854c7a58d2 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:26.697890 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6df41-7abd-43be-b298-6b41ede16b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:26.698807 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6df41-7abd-43be-b298-6b41ede16b5e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:26.699290 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6df41-7abd-43be-b298-6b41ede16b5e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:26.699753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6df41-7abd-43be-b298-6b41ede16b5e None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:26.700716 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6df41-7abd-43be-b298-6b41ede16b5e None admin] RBAC: Adding query filter params (name=member) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:26.701211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6df41-7abd-43be-b298-6b41ede16b5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:26.703642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6df41-7abd-43be-b298-6b41ede16b5e None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:26.712276 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 90/172] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:26.713364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6df41-7abd-43be-b298-6b41ede16b5e None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:26.733036 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3051a-e663-44cd-b6a7-cc88234a399c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:26.733790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3051a-e663-44cd-b6a7-cc88234a399c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:26.734331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3051a-e663-44cd-b6a7-cc88234a399c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:26.734433 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3051a-e663-44cd-b6a7-cc88234a399c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:26.735333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3051a-e663-44cd-b6a7-cc88234a399c None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=37e07551c481458b8e025b20f23fd735, user.id=c49606bc956845408ea8db21c53f217a) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:26.735695 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3051a-e663-44cd-b6a7-cc88234a399c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:26.737194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3051a-e663-44cd-b6a7-cc88234a399c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:26.743795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 83/173] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:26 2022] GET /identity/v3/role_assignments?user.id=c49606bc956845408ea8db21c53f217a&scope.project.id=37e07551c481458b8e025b20f23fd735&role.id=e59113c0366049e889df3f55eed90c72 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:26.746513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3051a-e663-44cd-b6a7-cc88234a399c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:26.828635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6df41-7abd-43be-b298-6b41ede16b5e None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:26.871666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb58d9-1f91-48a3-9158-6078228a7fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:26.872912 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb58d9-1f91-48a3-9158-6078228a7fac None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:26.873335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb58d9-1f91-48a3-9158-6078228a7fac None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:26.873745 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb58d9-1f91-48a3-9158-6078228a7fac None admin] PATH_INFO: `/v3/users/666162c9a04948b295bb104f9f912b00` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:26.884880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb58d9-1f91-48a3-9158-6078228a7fac None admin] RBAC: Authorizing `identity:get_user(user_id=666162c9a04948b295bb104f9f912b00)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:26.886613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb58d9-1f91-48a3-9158-6078228a7fac None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:26.888698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 91/174] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:26 2022] GET /identity/v3/users/666162c9a04948b295bb104f9f912b00 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:26.894641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb58d9-1f91-48a3-9158-6078228a7fac None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:26.911920 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3051a-e663-44cd-b6a7-cc88234a399c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:26.973330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c849574-3d84-4513-827a-335b71685965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:26.974397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c849574-3d84-4513-827a-335b71685965 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:26.974723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c849574-3d84-4513-827a-335b71685965 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:26.980146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c849574-3d84-4513-827a-335b71685965 None admin] PATH_INFO: `/v3/users/3248cf2854434b85b81170d82133b435` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:26.981359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c849574-3d84-4513-827a-335b71685965 None admin] RBAC: Authorizing `identity:get_user(user_id=3248cf2854434b85b81170d82133b435)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:26.983092 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c849574-3d84-4513-827a-335b71685965 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:26.985440 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 84/175] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:26 2022] GET /identity/v3/users/3248cf2854434b85b81170d82133b435 => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:26.990672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c849574-3d84-4513-827a-335b71685965 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:27.059346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb58d9-1f91-48a3-9158-6078228a7fac None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:27.121963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1e5d2-84ff-48b2-ad1f-7a01e51dc15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:27.122844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1e5d2-84ff-48b2-ad1f-7a01e51dc15c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:27.123124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1e5d2-84ff-48b2-ad1f-7a01e51dc15c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:27.123398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1e5d2-84ff-48b2-ad1f-7a01e51dc15c None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:27.125484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1e5d2-84ff-48b2-ad1f-7a01e51dc15c None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:27.126921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1e5d2-84ff-48b2-ad1f-7a01e51dc15c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:27.128980 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 92/176] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:26 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:27.134473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1e5d2-84ff-48b2-ad1f-7a01e51dc15c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:27.148617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c849574-3d84-4513-827a-335b71685965 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:27.197565 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b39cb-3fa9-4f71-ad95-8bca5d5da1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:27.198573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b39cb-3fa9-4f71-ad95-8bca5d5da1bc None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:27.199072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b39cb-3fa9-4f71-ad95-8bca5d5da1bc None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:27.199561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b39cb-3fa9-4f71-ad95-8bca5d5da1bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:27.237708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1e5d2-84ff-48b2-ad1f-7a01e51dc15c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:27.275497 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2d1df-45db-4bfe-87cb-9bb66e3ba621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:27.279981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2d1df-45db-4bfe-87cb-9bb66e3ba621 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:27.279981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2d1df-45db-4bfe-87cb-9bb66e3ba621 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:27.279981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2d1df-45db-4bfe-87cb-9bb66e3ba621 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735/users/666162c9a04948b295bb104f9f912b00/roles/2a69f0e80033468f96c13ef5f64d094d` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:27.279981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2d1df-45db-4bfe-87cb-9bb66e3ba621 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e07551c481458b8e025b20f23fd735, user_id=666162c9a04948b295bb104f9f912b00, role_id=2a69f0e80033468f96c13ef5f64d094d)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:27.280965 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2d1df-45db-4bfe-87cb-9bb66e3ba621 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:27.294849 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b39cb-3fa9-4f71-ad95-8bca5d5da1bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:27.297207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b39cb-3fa9-4f71-ad95-8bca5d5da1bc None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:27.301168 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 93/177] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:12:27 2022] PUT /identity/v3/projects/37e07551c481458b8e025b20f23fd735/users/666162c9a04948b295bb104f9f912b00/roles/2a69f0e80033468f96c13ef5f64d094d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:12:27.413280 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 85/178] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 04:12:27.419551 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2d1df-45db-4bfe-87cb-9bb66e3ba621 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:27.570902 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2d1df-45db-4bfe-87cb-9bb66e3ba621 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:27.605037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76cc5e-ed97-4ac7-ae16-cd4361d52299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:27.605912 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76cc5e-ed97-4ac7-ae16-cd4361d52299 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:27.606197 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76cc5e-ed97-4ac7-ae16-cd4361d52299 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:27.606471 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76cc5e-ed97-4ac7-ae16-cd4361d52299 None admin] PATH_INFO: `/v3/projects/a46b3ac7bb684582919f66ef0ee84b78` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:27.607656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76cc5e-ed97-4ac7-ae16-cd4361d52299 None admin] RBAC: Authorizing `identity:get_project(project_id=a46b3ac7bb684582919f66ef0ee84b78)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:27.609817 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76cc5e-ed97-4ac7-ae16-cd4361d52299 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:27.611676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 94/179] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:27 2022] GET /identity/v3/projects/a46b3ac7bb684582919f66ef0ee84b78 => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:27.617315 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b39cb-3fa9-4f71-ad95-8bca5d5da1bc None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:27.775959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b39cb-3fa9-4f71-ad95-8bca5d5da1bc None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:27.815292 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec64ed3-56fa-4c0d-9f3f-9786e8b27c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:27.816381 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec64ed3-56fa-4c0d-9f3f-9786e8b27c80 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:27.816861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec64ed3-56fa-4c0d-9f3f-9786e8b27c80 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:27.817508 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec64ed3-56fa-4c0d-9f3f-9786e8b27c80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:27.818536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec64ed3-56fa-4c0d-9f3f-9786e8b27c80 None admin] RBAC: Adding query filter params (role.id=d8b94102152a46768d568f66794854f3, scope.project.id=a46b3ac7bb684582919f66ef0ee84b78, user.id=3248cf2854434b85b81170d82133b435) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:27.819050 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec64ed3-56fa-4c0d-9f3f-9786e8b27c80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:27.820711 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec64ed3-56fa-4c0d-9f3f-9786e8b27c80 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:27.828386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 86/180] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:27 2022] GET /identity/v3/role_assignments?user.id=3248cf2854434b85b81170d82133b435&scope.project.id=a46b3ac7bb684582919f66ef0ee84b78&role.id=d8b94102152a46768d568f66794854f3 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:28.790449 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd03c0c-f245-4e67-88f3-220a4b238705 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:28.791420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd03c0c-f245-4e67-88f3-220a4b238705 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:28.791963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd03c0c-f245-4e67-88f3-220a4b238705 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:28.793771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 95/181] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:28.799390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14b49e-1202-4826-99b6-a8d26571cc10 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:28.800278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14b49e-1202-4826-99b6-a8d26571cc10 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:28.800756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14b49e-1202-4826-99b6-a8d26571cc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:28.819414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-0d14b49e-1202-4826-99b6-a8d26571cc10 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:28.821035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0d14b49e-1202-4826-99b6-a8d26571cc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:28.893922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 87/182] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:28.917289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd03c0c-f245-4e67-88f3-220a4b238705 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:29.012029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edc964-6a0f-4733-84eb-ad6ef34e77d7 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:29.012675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edc964-6a0f-4733-84eb-ad6ef34e77d7 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:29.013162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edc964-6a0f-4733-84eb-ad6ef34e77d7 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:29.014825 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 88/183] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:12:29.019867 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80fe7a-db46-4072-9ef0-017994429fbe None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:29.020395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80fe7a-db46-4072-9ef0-017994429fbe None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:29.020598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd03c0c-f245-4e67-88f3-220a4b238705 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:29.021079 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80fe7a-db46-4072-9ef0-017994429fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:29.041590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-9e80fe7a-db46-4072-9ef0-017994429fbe None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:29.043716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9e80fe7a-db46-4072-9ef0-017994429fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:29.053790 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd5389-9b01-43dc-8fe2-2b40c0f453cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:29.054577 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd5389-9b01-43dc-8fe2-2b40c0f453cd None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:29.055032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd5389-9b01-43dc-8fe2-2b40c0f453cd None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:29.055322 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd5389-9b01-43dc-8fe2-2b40c0f453cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:29.066050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd5389-9b01-43dc-8fe2-2b40c0f453cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-41dd5389-9b01-43dc-8fe2-2b40c0f453cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:29.073717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:29.074620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:29.075410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:29.075410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:29.075410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:29.075410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:29.075410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:29.075410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:29.075410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:29.075410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:29.075410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:29.075410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:29.075410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:29.075410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:29.075410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 96/184] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:12:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:29.081556 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd5389-9b01-43dc-8fe2-2b40c0f453cd None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:29.126995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 89/185] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:29.151217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80fe7a-db46-4072-9ef0-017994429fbe None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:29.193751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd5389-9b01-43dc-8fe2-2b40c0f453cd None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:29.227641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12e8e1-c2d0-4aa2-b5b7-018a8d200d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:29.228702 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12e8e1-c2d0-4aa2-b5b7-018a8d200d66 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:29.229184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12e8e1-c2d0-4aa2-b5b7-018a8d200d66 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:29.229652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12e8e1-c2d0-4aa2-b5b7-018a8d200d66 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:29.230623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12e8e1-c2d0-4aa2-b5b7-018a8d200d66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:29.231122 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12e8e1-c2d0-4aa2-b5b7-018a8d200d66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:29.232738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12e8e1-c2d0-4aa2-b5b7-018a8d200d66 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:29.242489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 97/186] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:29.250873 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12e8e1-c2d0-4aa2-b5b7-018a8d200d66 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:29.265769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80fe7a-db46-4072-9ef0-017994429fbe None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:29.300511 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428f640-35ca-4b14-b430-140937718361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:29.301655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428f640-35ca-4b14-b430-140937718361 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:29.302139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428f640-35ca-4b14-b430-140937718361 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:29.302609 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428f640-35ca-4b14-b430-140937718361 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:29.304037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428f640-35ca-4b14-b430-140937718361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:29.305709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428f640-35ca-4b14-b430-140937718361 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:29.312234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 90/187] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:29.313718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dda84b-80ed-42c4-aeb1-ea2e1f60d62e None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:29.314006 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dda84b-80ed-42c4-aeb1-ea2e1f60d62e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:29.314283 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dda84b-80ed-42c4-aeb1-ea2e1f60d62e None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:29.319581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 91/188] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:29.319581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dda84b-80ed-42c4-aeb1-ea2e1f60d62e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:29.350744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12e8e1-c2d0-4aa2-b5b7-018a8d200d66 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:29.383428 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f8f03-11a5-4b2a-a27d-39bfde4b93b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:29.384331 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f8f03-11a5-4b2a-a27d-39bfde4b93b3 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:29.384615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f8f03-11a5-4b2a-a27d-39bfde4b93b3 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:29.384894 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f8f03-11a5-4b2a-a27d-39bfde4b93b3 None admin] PATH_INFO: `/v3/users/9014bf749deb446091ec762123c0d404` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:29.398545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f8f03-11a5-4b2a-a27d-39bfde4b93b3 None admin] RBAC: Authorizing `identity:get_user(user_id=9014bf749deb446091ec762123c0d404)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:29.400075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f8f03-11a5-4b2a-a27d-39bfde4b93b3 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:29.401742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 98/189] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:29 2022] GET /identity/v3/users/9014bf749deb446091ec762123c0d404 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:29.403205 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0af529-1a6f-4e2c-9b6d-33d2b5e50daf None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:29.403500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0af529-1a6f-4e2c-9b6d-33d2b5e50daf None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:29.403764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0af529-1a6f-4e2c-9b6d-33d2b5e50daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:29.420589 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-da0af529-1a6f-4e2c-9b6d-33d2b5e50daf None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:29.422488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-da0af529-1a6f-4e2c-9b6d-33d2b5e50daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:29.423494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dda84b-80ed-42c4-aeb1-ea2e1f60d62e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:29.455874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebaf39b-fed3-4f3d-a904-8820576ac801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:29.456575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebaf39b-fed3-4f3d-a904-8820576ac801 None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:29.456853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebaf39b-fed3-4f3d-a904-8820576ac801 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:29.457120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebaf39b-fed3-4f3d-a904-8820576ac801 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:29.457810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebaf39b-fed3-4f3d-a904-8820576ac801 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:29.459227 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebaf39b-fed3-4f3d-a904-8820576ac801 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:29.479035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 92/190] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:12:29 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:12:29.488419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 99/191] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:29.489462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebaf39b-fed3-4f3d-a904-8820576ac801 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:29.511857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0af529-1a6f-4e2c-9b6d-33d2b5e50daf None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:29.622300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebaf39b-fed3-4f3d-a904-8820576ac801 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:29.637734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0af529-1a6f-4e2c-9b6d-33d2b5e50daf None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:29.665228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba7302-c53f-440b-838d-14cbf81a0231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:29.665923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba7302-c53f-440b-838d-14cbf81a0231 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:29.666205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba7302-c53f-440b-838d-14cbf81a0231 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:29.666582 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba7302-c53f-440b-838d-14cbf81a0231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:29.681373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a1983-243a-4dad-86f4-0dc35811fa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:29.682713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1983-243a-4dad-86f4-0dc35811fa92 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:29.683242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1983-243a-4dad-86f4-0dc35811fa92 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:29.683689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1983-243a-4dad-86f4-0dc35811fa92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:29.697921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a1983-243a-4dad-86f4-0dc35811fa92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-ce3a1983-243a-4dad-86f4-0dc35811fa92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:29.706839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:29.707793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:29.708615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:29.708615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:29.708615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:29.708615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:29.708615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:29.708615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:29.708615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:29.708615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:29.708615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:29.708615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:29.708615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:29.708615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:29.714468 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 100/192] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:12:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:12:29.716076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a1983-243a-4dad-86f4-0dc35811fa92 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:29.762360 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba7302-c53f-440b-838d-14cbf81a0231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:29.763946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba7302-c53f-440b-838d-14cbf81a0231 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:29.834686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a1983-243a-4dad-86f4-0dc35811fa92 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:29.868457 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17788f-a1c0-4604-8b2a-140824d788da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:29.869128 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17788f-a1c0-4604-8b2a-140824d788da None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:29.869407 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17788f-a1c0-4604-8b2a-140824d788da None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:29.869679 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17788f-a1c0-4604-8b2a-140824d788da None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:29.870558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17788f-a1c0-4604-8b2a-140824d788da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:29.870732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17788f-a1c0-4604-8b2a-140824d788da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:29.872128 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17788f-a1c0-4604-8b2a-140824d788da None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:29.878143 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 93/193] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:29.881494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 101/194] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:12:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:29.884638 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba7302-c53f-440b-838d-14cbf81a0231 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:29.887464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17788f-a1c0-4604-8b2a-140824d788da None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:29.986645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba7302-c53f-440b-838d-14cbf81a0231 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:29.989109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17788f-a1c0-4604-8b2a-140824d788da None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:30.020209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32e52e-e69a-41bc-b14f-b6c261e3e483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:30.021281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32e52e-e69a-41bc-b14f-b6c261e3e483 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:30.021564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32e52e-e69a-41bc-b14f-b6c261e3e483 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:30.021844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32e52e-e69a-41bc-b14f-b6c261e3e483 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:30.022141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e341c-469d-4569-97c2-170105f5bc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:30.023021 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e341c-469d-4569-97c2-170105f5bc3e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:30.023141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32e52e-e69a-41bc-b14f-b6c261e3e483 None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:30.023298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e341c-469d-4569-97c2-170105f5bc3e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:30.023557 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e341c-469d-4569-97c2-170105f5bc3e None admin] PATH_INFO: `/v3/users/9014bf749deb446091ec762123c0d404` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:30.024488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32e52e-e69a-41bc-b14f-b6c261e3e483 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:30.024775 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e341c-469d-4569-97c2-170105f5bc3e None admin] RBAC: Authorizing `identity:get_user(user_id=9014bf749deb446091ec762123c0d404)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:30.026350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 94/195] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:29 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:30.026456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e341c-469d-4569-97c2-170105f5bc3e None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:30.027369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9585216-a8ff-4b62-b4fc-bb4d3e8206c3 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:30.027928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9585216-a8ff-4b62-b4fc-bb4d3e8206c3 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:30.028031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 102/196] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:29 2022] GET /identity/v3/users/9014bf749deb446091ec762123c0d404 => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:30.028118 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9585216-a8ff-4b62-b4fc-bb4d3e8206c3 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:30.029163 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 95/197] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:30.033722 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e341c-469d-4569-97c2-170105f5bc3e None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:30.034956 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9585216-a8ff-4b62-b4fc-bb4d3e8206c3 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:30.135259 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e341c-469d-4569-97c2-170105f5bc3e None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:30.136642 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9585216-a8ff-4b62-b4fc-bb4d3e8206c3 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:30.169300 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb2685-a924-4545-b9d1-ec58bb61617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:30.169951 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb2685-a924-4545-b9d1-ec58bb61617d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:30.170230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb2685-a924-4545-b9d1-ec58bb61617d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:30.170504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb2685-a924-4545-b9d1-ec58bb61617d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:30.172159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629aebb-8aed-4a76-8882-89824f2f92e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:30.172890 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629aebb-8aed-4a76-8882-89824f2f92e8 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:30.173263 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629aebb-8aed-4a76-8882-89824f2f92e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:30.173544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629aebb-8aed-4a76-8882-89824f2f92e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:30.174380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629aebb-8aed-4a76-8882-89824f2f92e8 None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f, user.id=9014bf749deb446091ec762123c0d404) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:30.174688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629aebb-8aed-4a76-8882-89824f2f92e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:30.176164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629aebb-8aed-4a76-8882-89824f2f92e8 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:30.182142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 96/198] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:30 2022] GET /identity/v3/role_assignments?user.id=9014bf749deb446091ec762123c0d404&scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f&role.id=e59113c0366049e889df3f55eed90c72 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:30.183409 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f577c2-106e-446c-a9e5-2121920b7833 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:30.183697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f577c2-106e-446c-a9e5-2121920b7833 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:30.184002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f577c2-106e-446c-a9e5-2121920b7833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:30.201783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-52f577c2-106e-446c-a9e5-2121920b7833 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:30.203858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-52f577c2-106e-446c-a9e5-2121920b7833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:30.276000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 97/199] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:30.300400 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f577c2-106e-446c-a9e5-2121920b7833 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:30.324926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb2685-a924-4545-b9d1-ec58bb61617d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:30.326445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb2685-a924-4545-b9d1-ec58bb61617d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:30.453880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f577c2-106e-446c-a9e5-2121920b7833 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:30.498599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cdce0-845c-437e-bcac-999d0658f226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:30.500553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cdce0-845c-437e-bcac-999d0658f226 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:30.504219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cdce0-845c-437e-bcac-999d0658f226 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:30.504687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cdce0-845c-437e-bcac-999d0658f226 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:30.527318 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cdce0-845c-437e-bcac-999d0658f226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:30.534974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 103/200] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-293cdce0-845c-437e-bcac-999d0658f226 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:30.542655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:30.543547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:30.544600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:30.544600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:30.544600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:30.544600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:30.544600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:30.544600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:30.544600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:30.544600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:30.544600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:30.544600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:30.544600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:30.544600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:30.546943 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 98/201] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:12:30.552919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cdce0-845c-437e-bcac-999d0658f226 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:30.553054 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb2685-a924-4545-b9d1-ec58bb61617d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:30.661552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cdce0-845c-437e-bcac-999d0658f226 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:30.699395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c385a44-b08f-43c4-914e-b7afa55564dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:30.700778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c385a44-b08f-43c4-914e-b7afa55564dd None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:30.701060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c385a44-b08f-43c4-914e-b7afa55564dd None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:30.701337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c385a44-b08f-43c4-914e-b7afa55564dd None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:30.702611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c385a44-b08f-43c4-914e-b7afa55564dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:30.702916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c385a44-b08f-43c4-914e-b7afa55564dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:30.704627 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c385a44-b08f-43c4-914e-b7afa55564dd None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:30.737071 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 99/202] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:30.741438 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c385a44-b08f-43c4-914e-b7afa55564dd None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:30.768635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb2685-a924-4545-b9d1-ec58bb61617d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:30.806900 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55d7ac-edc8-4832-86c2-3629bdf73916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:30.807771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55d7ac-edc8-4832-86c2-3629bdf73916 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:30.808865 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55d7ac-edc8-4832-86c2-3629bdf73916 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:30.809769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55d7ac-edc8-4832-86c2-3629bdf73916 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:30.816082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55d7ac-edc8-4832-86c2-3629bdf73916 None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:30.821749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55d7ac-edc8-4832-86c2-3629bdf73916 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:30.823727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 104/203] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:30 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:30.831599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55d7ac-edc8-4832-86c2-3629bdf73916 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:30.889175 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c385a44-b08f-43c4-914e-b7afa55564dd None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:30.937102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d6288-4d8a-4693-95d0-742614ec0851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:30.938339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d6288-4d8a-4693-95d0-742614ec0851 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:30.938818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d6288-4d8a-4693-95d0-742614ec0851 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:30.939284 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d6288-4d8a-4693-95d0-742614ec0851 None admin] PATH_INFO: `/v3/users/666162c9a04948b295bb104f9f912b00` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:30.940763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d6288-4d8a-4693-95d0-742614ec0851 None admin] RBAC: Authorizing `identity:get_user(user_id=666162c9a04948b295bb104f9f912b00)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:30.942432 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d6288-4d8a-4693-95d0-742614ec0851 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:30.944304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 100/204] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:30 2022] GET /identity/v3/users/666162c9a04948b295bb104f9f912b00 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:30.952802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d6288-4d8a-4693-95d0-742614ec0851 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:30.971238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55d7ac-edc8-4832-86c2-3629bdf73916 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:31.023549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a8c12-6747-4de8-b1de-944692b2e084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:31.024239 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a8c12-6747-4de8-b1de-944692b2e084 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:31.024520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a8c12-6747-4de8-b1de-944692b2e084 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:31.024792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a8c12-6747-4de8-b1de-944692b2e084 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:31.025678 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a8c12-6747-4de8-b1de-944692b2e084 None admin] RBAC: Adding query filter params (role.id=7600100dce614d9bb4d47b1e660b7bb2, scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f, user.id=9014bf749deb446091ec762123c0d404) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:31.025991 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a8c12-6747-4de8-b1de-944692b2e084 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:31.027490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a8c12-6747-4de8-b1de-944692b2e084 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:31.034877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 105/205] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:30 2022] GET /identity/v3/role_assignments?user.id=9014bf749deb446091ec762123c0d404&scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f&role.id=7600100dce614d9bb4d47b1e660b7bb2 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:31.081565 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d6288-4d8a-4693-95d0-742614ec0851 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:31.123018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b24b9e-2828-495c-af60-985c917ae983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:31.123698 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b24b9e-2828-495c-af60-985c917ae983 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:31.124558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b24b9e-2828-495c-af60-985c917ae983 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:31.124931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b24b9e-2828-495c-af60-985c917ae983 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:31.227680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b24b9e-2828-495c-af60-985c917ae983 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:31.229117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b24b9e-2828-495c-af60-985c917ae983 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:31.331922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 101/206] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:31.340989 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a8c12-6747-4de8-b1de-944692b2e084 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:31.475456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a8c12-6747-4de8-b1de-944692b2e084 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:31.522991 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556ee54-ab11-4220-89fe-fcef53dec24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:31.523849 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556ee54-ab11-4220-89fe-fcef53dec24b None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:31.524157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556ee54-ab11-4220-89fe-fcef53dec24b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:31.524433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556ee54-ab11-4220-89fe-fcef53dec24b None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:31.525644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556ee54-ab11-4220-89fe-fcef53dec24b None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:31.527078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556ee54-ab11-4220-89fe-fcef53dec24b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:31.528965 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 106/207] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:31 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:31.535080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b24b9e-2828-495c-af60-985c917ae983 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:31.703604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b24b9e-2828-495c-af60-985c917ae983 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:31.753610 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5fa90-d6f2-45c9-80f9-381c3d131e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:31.754279 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5fa90-d6f2-45c9-80f9-381c3d131e80 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:31.754594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5fa90-d6f2-45c9-80f9-381c3d131e80 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:31.754834 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5fa90-d6f2-45c9-80f9-381c3d131e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:31.755679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5fa90-d6f2-45c9-80f9-381c3d131e80 None admin] RBAC: Adding query filter params (role.id=2a69f0e80033468f96c13ef5f64d094d, scope.project.id=37e07551c481458b8e025b20f23fd735, user.id=666162c9a04948b295bb104f9f912b00) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:31.756016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5fa90-d6f2-45c9-80f9-381c3d131e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:31.757476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5fa90-d6f2-45c9-80f9-381c3d131e80 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:31.768753 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 102/208] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:31 2022] GET /identity/v3/role_assignments?user.id=666162c9a04948b295bb104f9f912b00&scope.project.id=37e07551c481458b8e025b20f23fd735&role.id=2a69f0e80033468f96c13ef5f64d094d => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:32.277264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a36dc-8f46-414a-b6fd-3cf99c0b0331 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:32.278258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a36dc-8f46-414a-b6fd-3cf99c0b0331 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:32.278737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a36dc-8f46-414a-b6fd-3cf99c0b0331 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:32.280582 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 107/209] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:32.286452 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fa1b0-0e8c-457c-90ee-632f20ee542e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:32.287317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fa1b0-0e8c-457c-90ee-632f20ee542e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:32.287787 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fa1b0-0e8c-457c-90ee-632f20ee542e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:32.308580 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-295fa1b0-0e8c-457c-90ee-632f20ee542e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:32.310148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-295fa1b0-0e8c-457c-90ee-632f20ee542e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:32.393292 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 103/210] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:32.409674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d50b70-e61c-4e26-823e-ea24535dd414 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:32.410277 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d50b70-e61c-4e26-823e-ea24535dd414 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:32.410749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d50b70-e61c-4e26-823e-ea24535dd414 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:32.412366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 108/211] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:32.418024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733275d-3f6b-496c-a5ed-01d54df471d0 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:32.418313 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733275d-3f6b-496c-a5ed-01d54df471d0 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:32.418594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733275d-3f6b-496c-a5ed-01d54df471d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:32.422697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d50b70-e61c-4e26-823e-ea24535dd414 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:32.443318 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-9733275d-3f6b-496c-a5ed-01d54df471d0 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:32.444931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9733275d-3f6b-496c-a5ed-01d54df471d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:32.523454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 104/212] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:32.548153 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733275d-3f6b-496c-a5ed-01d54df471d0 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:32.553416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d50b70-e61c-4e26-823e-ea24535dd414 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:32.597203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0298bc-5cd9-4aa5-b435-eb285e2082e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:32.598012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0298bc-5cd9-4aa5-b435-eb285e2082e8 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:32.598297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0298bc-5cd9-4aa5-b435-eb285e2082e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:32.598576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0298bc-5cd9-4aa5-b435-eb285e2082e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:32.612345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0298bc-5cd9-4aa5-b435-eb285e2082e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-0b0298bc-5cd9-4aa5-b435-eb285e2082e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:32.622208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:32.623127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:32.624554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:32.624554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:32.624554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:32.624554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:32.624554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:32.624554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:32.624554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:32.624554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:32.624554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:32.624554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:32.624554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:32.624554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:32.625150 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 109/213] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:32.632800 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0298bc-5cd9-4aa5-b435-eb285e2082e8 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:32.649957 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733275d-3f6b-496c-a5ed-01d54df471d0 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:32.690904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07ba30-6ffc-47ef-a494-94ba100f0fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:32.691935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07ba30-6ffc-47ef-a494-94ba100f0fca None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:32.692386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07ba30-6ffc-47ef-a494-94ba100f0fca None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:32.692761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07ba30-6ffc-47ef-a494-94ba100f0fca None admin] PATH_INFO: `/v3/domains/default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:32.694154 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07ba30-6ffc-47ef-a494-94ba100f0fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:32.695836 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07ba30-6ffc-47ef-a494-94ba100f0fca None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:32.697804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 105/214] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:32.703807 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07ba30-6ffc-47ef-a494-94ba100f0fca None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:32.762858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0298bc-5cd9-4aa5-b435-eb285e2082e8 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:32.801275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f8b38-ee23-4463-8697-b1aafdf06205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:32.802131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f8b38-ee23-4463-8697-b1aafdf06205 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:32.802571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f8b38-ee23-4463-8697-b1aafdf06205 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:32.802948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f8b38-ee23-4463-8697-b1aafdf06205 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:32.803802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f8b38-ee23-4463-8697-b1aafdf06205 None admin] RBAC: Adding query filter params (name=member) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:32.804237 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f8b38-ee23-4463-8697-b1aafdf06205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:32.805700 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f8b38-ee23-4463-8697-b1aafdf06205 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:32.810940 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07ba30-6ffc-47ef-a494-94ba100f0fca None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:32.819480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 110/215] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:32.820924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f8b38-ee23-4463-8697-b1aafdf06205 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:32.847705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76a144-5886-40d1-a537-dd122ae9cdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:32.848798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76a144-5886-40d1-a537-dd122ae9cdb2 None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:32.849274 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76a144-5886-40d1-a537-dd122ae9cdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:32.849740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76a144-5886-40d1-a537-dd122ae9cdb2 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:32.850643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76a144-5886-40d1-a537-dd122ae9cdb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:32.852286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76a144-5886-40d1-a537-dd122ae9cdb2 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:32.879822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 106/216] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:12:32 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:12:32.882282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be495646-1a76-4fda-a9b7-b05a79d190c9 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:32.883116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be495646-1a76-4fda-a9b7-b05a79d190c9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:32.883862 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be495646-1a76-4fda-a9b7-b05a79d190c9 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:32.886198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 107/217] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:32.892852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447c30-f289-44bb-b210-117e1dde78be None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:32.893500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447c30-f289-44bb-b210-117e1dde78be None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:32.894607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a447c30-f289-44bb-b210-117e1dde78be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:32.919016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-4a447c30-f289-44bb-b210-117e1dde78be None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:32.921811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-4a447c30-f289-44bb-b210-117e1dde78be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:32.933186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f8b38-ee23-4463-8697-b1aafdf06205 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:32.978852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f54e71-abec-4950-b65e-3998f03a3014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:32.980177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f54e71-abec-4950-b65e-3998f03a3014 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:32.980735 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f54e71-abec-4950-b65e-3998f03a3014 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:32.981206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f54e71-abec-4950-b65e-3998f03a3014 None admin] PATH_INFO: `/v3/users/b857f9f672f1497cad7c9e114cb66e80` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:33.003048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f54e71-abec-4950-b65e-3998f03a3014 None admin] RBAC: Authorizing `identity:get_user(user_id=b857f9f672f1497cad7c9e114cb66e80)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:33.005692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f54e71-abec-4950-b65e-3998f03a3014 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:33.012584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 111/218] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:32 2022] GET /identity/v3/users/b857f9f672f1497cad7c9e114cb66e80 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:33.014704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 108/219] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:33.015262 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f54e71-abec-4950-b65e-3998f03a3014 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:33.038864 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a447c30-f289-44bb-b210-117e1dde78be None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:33.146851 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f54e71-abec-4950-b65e-3998f03a3014 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:33.167776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a447c30-f289-44bb-b210-117e1dde78be None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:33.191385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb85ee-a5c6-4fcb-9d6d-692e9e15b375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:33.192086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb85ee-a5c6-4fcb-9d6d-692e9e15b375 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:33.192377 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb85ee-a5c6-4fcb-9d6d-692e9e15b375 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:33.192645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb85ee-a5c6-4fcb-9d6d-692e9e15b375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:33.214043 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0dc91-ed91-43c8-bb17-ce8d9d74b3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:33.214845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0dc91-ed91-43c8-bb17-ce8d9d74b3ce None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:33.215128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0dc91-ed91-43c8-bb17-ce8d9d74b3ce None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:33.215402 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0dc91-ed91-43c8-bb17-ce8d9d74b3ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:33.231187 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0dc91-ed91-43c8-bb17-ce8d9d74b3ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-69c0dc91-ed91-43c8-bb17-ce8d9d74b3ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:33.242151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:33.243072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:33.243839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:33.243839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:33.243839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:33.243839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:33.243839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:33.243839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:33.243839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:33.243839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:33.243839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:33.243839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:33.243839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:33.243839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:33.244521 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 109/220] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:12:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:33.250650 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0dc91-ed91-43c8-bb17-ce8d9d74b3ce None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:33.276432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb85ee-a5c6-4fcb-9d6d-692e9e15b375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:33.278702 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb85ee-a5c6-4fcb-9d6d-692e9e15b375 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:33.369720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0dc91-ed91-43c8-bb17-ce8d9d74b3ce None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:33.383015 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 112/221] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:33.388617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb85ee-a5c6-4fcb-9d6d-692e9e15b375 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:33.406723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35ad77-db55-406b-a3dc-041699ecf3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:33.407323 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35ad77-db55-406b-a3dc-041699ecf3f9 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:33.407606 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35ad77-db55-406b-a3dc-041699ecf3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:33.407884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35ad77-db55-406b-a3dc-041699ecf3f9 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:33.408670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35ad77-db55-406b-a3dc-041699ecf3f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:33.408974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35ad77-db55-406b-a3dc-041699ecf3f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:33.410370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35ad77-db55-406b-a3dc-041699ecf3f9 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:33.419413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 110/222] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:33.426576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35ad77-db55-406b-a3dc-041699ecf3f9 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:33.489408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb85ee-a5c6-4fcb-9d6d-692e9e15b375 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:33.523928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de907d6-4ab7-437f-a91e-f792446650d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:33.524814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de907d6-4ab7-437f-a91e-f792446650d6 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:33.525089 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de907d6-4ab7-437f-a91e-f792446650d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:33.525364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de907d6-4ab7-437f-a91e-f792446650d6 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:33.526568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35ad77-db55-406b-a3dc-041699ecf3f9 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:33.526748 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de907d6-4ab7-437f-a91e-f792446650d6 None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:33.528361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de907d6-4ab7-437f-a91e-f792446650d6 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:33.531086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 113/223] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:33 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:33.536835 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de907d6-4ab7-437f-a91e-f792446650d6 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:33.562018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba551356-eafd-4941-b037-47bdad1cf1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:33.562903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba551356-eafd-4941-b037-47bdad1cf1f5 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:33.563187 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba551356-eafd-4941-b037-47bdad1cf1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:33.563464 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba551356-eafd-4941-b037-47bdad1cf1f5 None admin] PATH_INFO: `/v3/users/9014bf749deb446091ec762123c0d404` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:33.564693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba551356-eafd-4941-b037-47bdad1cf1f5 None admin] RBAC: Authorizing `identity:get_user(user_id=9014bf749deb446091ec762123c0d404)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:33.566126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba551356-eafd-4941-b037-47bdad1cf1f5 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:33.567748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 111/224] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:33 2022] GET /identity/v3/users/9014bf749deb446091ec762123c0d404 => generated 283 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:33.573313 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba551356-eafd-4941-b037-47bdad1cf1f5 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:33.638104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de907d6-4ab7-437f-a91e-f792446650d6 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:33.673290 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c68c73-ab7f-4655-a2bb-fcf5c7a797ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:33.673915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c68c73-ab7f-4655-a2bb-fcf5c7a797ad None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:33.674199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c68c73-ab7f-4655-a2bb-fcf5c7a797ad None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:33.674475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c68c73-ab7f-4655-a2bb-fcf5c7a797ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:33.675309 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c68c73-ab7f-4655-a2bb-fcf5c7a797ad None admin] RBAC: Adding query filter params (role.id=d8b94102152a46768d568f66794854f3, scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f, user.id=b857f9f672f1497cad7c9e114cb66e80) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:33.675620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c68c73-ab7f-4655-a2bb-fcf5c7a797ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:33.676477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba551356-eafd-4941-b037-47bdad1cf1f5 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:33.677091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c68c73-ab7f-4655-a2bb-fcf5c7a797ad None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:33.682969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 114/225] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:33 2022] GET /identity/v3/role_assignments?user.id=b857f9f672f1497cad7c9e114cb66e80&scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f&role.id=d8b94102152a46768d568f66794854f3 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:33.692488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e4586-8573-4038-b3e6-28c395945083 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:33.692800 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e4586-8573-4038-b3e6-28c395945083 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:33.693048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e4586-8573-4038-b3e6-28c395945083 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:33.694774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 115/226] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:33.700629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e61e71-ba9f-41ab-972b-c160b9c65964 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:33.700919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e61e71-ba9f-41ab-972b-c160b9c65964 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:33.701180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e61e71-ba9f-41ab-972b-c160b9c65964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:33.712617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46ebb-0c2b-4d45-b924-70a6e6a0d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:33.713502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46ebb-0c2b-4d45-b924-70a6e6a0d241 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:33.713785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46ebb-0c2b-4d45-b924-70a6e6a0d241 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:33.714062 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46ebb-0c2b-4d45-b924-70a6e6a0d241 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:33.715258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46ebb-0c2b-4d45-b924-70a6e6a0d241 None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:33.716737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46ebb-0c2b-4d45-b924-70a6e6a0d241 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:33.718954 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 112/227] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:33 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:33.724468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46ebb-0c2b-4d45-b924-70a6e6a0d241 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:33.735425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-35e61e71-ba9f-41ab-972b-c160b9c65964 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:33.736646 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-35e61e71-ba9f-41ab-972b-c160b9c65964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:33.829396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 116/228] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:33.832416 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46ebb-0c2b-4d45-b924-70a6e6a0d241 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:33.864390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e61e71-ba9f-41ab-972b-c160b9c65964 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:33.869494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d1c85-872d-43e8-b4be-9f8b50934e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:33.870420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d1c85-872d-43e8-b4be-9f8b50934e03 None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:33.870705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d1c85-872d-43e8-b4be-9f8b50934e03 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:33.870978 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d1c85-872d-43e8-b4be-9f8b50934e03 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f/users/9014bf749deb446091ec762123c0d404/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:33.873198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d1c85-872d-43e8-b4be-9f8b50934e03 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f, user_id=9014bf749deb446091ec762123c0d404, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:33.874734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d1c85-872d-43e8-b4be-9f8b50934e03 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:33.911338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 113/229] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:12:33 2022] PUT /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f/users/9014bf749deb446091ec762123c0d404/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:12:34.040220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e61e71-ba9f-41ab-972b-c160b9c65964 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:34.089636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fc1a1-8c5f-47d9-8c8d-f23ae6ec3f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:34.090534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fc1a1-8c5f-47d9-8c8d-f23ae6ec3f91 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:34.090822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fc1a1-8c5f-47d9-8c8d-f23ae6ec3f91 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:34.091096 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fc1a1-8c5f-47d9-8c8d-f23ae6ec3f91 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:34.106518 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fc1a1-8c5f-47d9-8c8d-f23ae6ec3f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-2e8fc1a1-8c5f-47d9-8c8d-f23ae6ec3f91 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:34.115376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:34.116367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:34.117138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:34.117138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:34.117138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:34.117138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:34.117138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:34.117138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:34.117138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:34.117138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:34.117138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:34.117138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:34.117138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:34.117138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:34.119268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 117/230] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:12:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:12:34.125947 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d1c85-872d-43e8-b4be-9f8b50934e03 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:34.274291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d1c85-872d-43e8-b4be-9f8b50934e03 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:34.326534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e88431-8d3f-4351-b647-ae9f935ebe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:34.327746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e88431-8d3f-4351-b647-ae9f935ebe24 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:34.328068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e88431-8d3f-4351-b647-ae9f935ebe24 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:34.328588 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e88431-8d3f-4351-b647-ae9f935ebe24 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:34.329349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e88431-8d3f-4351-b647-ae9f935ebe24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:34.329654 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e88431-8d3f-4351-b647-ae9f935ebe24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:34.331073 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e88431-8d3f-4351-b647-ae9f935ebe24 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:34.345210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 114/231] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:12:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:34.357033 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fc1a1-8c5f-47d9-8c8d-f23ae6ec3f91 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:34.479415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fc1a1-8c5f-47d9-8c8d-f23ae6ec3f91 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:34.518934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa133b-67ff-466b-aa07-aea14e4a3224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:34.519822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa133b-67ff-466b-aa07-aea14e4a3224 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:34.520128 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa133b-67ff-466b-aa07-aea14e4a3224 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:34.520400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa133b-67ff-466b-aa07-aea14e4a3224 None admin] PATH_INFO: `/v3/users/9014bf749deb446091ec762123c0d404` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:34.524618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa133b-67ff-466b-aa07-aea14e4a3224 None admin] RBAC: Authorizing `identity:get_user(user_id=9014bf749deb446091ec762123c0d404)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:34.526091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa133b-67ff-466b-aa07-aea14e4a3224 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:34.528055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 118/232] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:34 2022] GET /identity/v3/users/9014bf749deb446091ec762123c0d404 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:34.533574 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e88431-8d3f-4351-b647-ae9f935ebe24 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:34.672913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e88431-8d3f-4351-b647-ae9f935ebe24 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:34.718077 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c86c45-e70b-4985-bdba-566f2e364929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:34.718969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c86c45-e70b-4985-bdba-566f2e364929 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:34.719248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c86c45-e70b-4985-bdba-566f2e364929 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:34.719519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c86c45-e70b-4985-bdba-566f2e364929 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:34.720759 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c86c45-e70b-4985-bdba-566f2e364929 None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:34.722215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c86c45-e70b-4985-bdba-566f2e364929 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:34.724246 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 115/233] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:34 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:34.729787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa133b-67ff-466b-aa07-aea14e4a3224 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:34.921456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa133b-67ff-466b-aa07-aea14e4a3224 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:34.960507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4dd2a-bad3-4568-9b19-2ac2ca67b938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:34.961747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4dd2a-bad3-4568-9b19-2ac2ca67b938 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:34.962221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4dd2a-bad3-4568-9b19-2ac2ca67b938 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:34.962699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4dd2a-bad3-4568-9b19-2ac2ca67b938 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f/users/9014bf749deb446091ec762123c0d404/roles/7600100dce614d9bb4d47b1e660b7bb2` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:34.965138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4dd2a-bad3-4568-9b19-2ac2ca67b938 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f, user_id=9014bf749deb446091ec762123c0d404, role_id=7600100dce614d9bb4d47b1e660b7bb2)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:34.966770 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4dd2a-bad3-4568-9b19-2ac2ca67b938 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:35.007201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 119/234] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:12:34 2022] PUT /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f/users/9014bf749deb446091ec762123c0d404/roles/7600100dce614d9bb4d47b1e660b7bb2 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:12:35.612058 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb65a-64ae-4b69-bbff-5834944c3723 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:35.612617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb65a-64ae-4b69-bbff-5834944c3723 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:35.612617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfb65a-64ae-4b69-bbff-5834944c3723 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:35.614115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 116/235] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:35.619646 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1998f0-86f1-4804-8128-370da6aa041b None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:35.620087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1998f0-86f1-4804-8128-370da6aa041b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:35.620279 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1998f0-86f1-4804-8128-370da6aa041b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:35.644859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-ea1998f0-86f1-4804-8128-370da6aa041b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:35.646038 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ea1998f0-86f1-4804-8128-370da6aa041b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:35.734308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 120/236] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:35.762623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfb65a-64ae-4b69-bbff-5834944c3723 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:35.878477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfb65a-64ae-4b69-bbff-5834944c3723 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:35.912567 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7c29e-669d-47a4-aeef-35067db7849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:35.913376 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7c29e-669d-47a4-aeef-35067db7849a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:35.913659 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7c29e-669d-47a4-aeef-35067db7849a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:35.913936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7c29e-669d-47a4-aeef-35067db7849a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:35.924667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7c29e-669d-47a4-aeef-35067db7849a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-d6a7c29e-669d-47a4-aeef-35067db7849a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:35.932428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:35.933326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:35.934500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:35.934500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:35.934500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:35.934500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:35.934500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:35.934500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:35.934500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:35.934500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:35.934500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:35.934500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:35.934500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:35.934500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:35.934500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 117/237] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:35.939418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1998f0-86f1-4804-8128-370da6aa041b None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:35.989484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d59701-3d5f-45f4-a203-4f2411286f0e None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:35.989784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d59701-3d5f-45f4-a203-4f2411286f0e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:35.990037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d59701-3d5f-45f4-a203-4f2411286f0e None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:35.991715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 118/238] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:36.004900 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21869cbf-2b5b-4c8e-89df-cb58c8173e52 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:36.005211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21869cbf-2b5b-4c8e-89df-cb58c8173e52 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:36.005455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21869cbf-2b5b-4c8e-89df-cb58c8173e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:36.034200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-21869cbf-2b5b-4c8e-89df-cb58c8173e52 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:36.035391 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-21869cbf-2b5b-4c8e-89df-cb58c8173e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:36.042942 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1998f0-86f1-4804-8128-370da6aa041b None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:36.085603 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c669d-84d5-47fb-bcab-da24a9e4d454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:36.086675 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c669d-84d5-47fb-bcab-da24a9e4d454 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:36.087160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c669d-84d5-47fb-bcab-da24a9e4d454 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:36.087632 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c669d-84d5-47fb-bcab-da24a9e4d454 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:36.088719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c669d-84d5-47fb-bcab-da24a9e4d454 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:36.089229 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c669d-84d5-47fb-bcab-da24a9e4d454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:36.090811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c669d-84d5-47fb-bcab-da24a9e4d454 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:36.111922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 121/239] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:36.117269 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c669d-84d5-47fb-bcab-da24a9e4d454 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:36.124624 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 119/240] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:36.169004 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21869cbf-2b5b-4c8e-89df-cb58c8173e52 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:36.279042 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c669d-84d5-47fb-bcab-da24a9e4d454 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:36.323257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735bdc6b-9a37-4a53-bc10-3b7e4dc97ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:36.324168 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bdc6b-9a37-4a53-bc10-3b7e4dc97ca9 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:36.324451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bdc6b-9a37-4a53-bc10-3b7e4dc97ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:36.324727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bdc6b-9a37-4a53-bc10-3b7e4dc97ca9 None admin] PATH_INFO: `/v3/users/1ac46762402b426389ab68171593c437` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:36.353332 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21869cbf-2b5b-4c8e-89df-cb58c8173e52 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:36.355184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735bdc6b-9a37-4a53-bc10-3b7e4dc97ca9 None admin] RBAC: Authorizing `identity:get_user(user_id=1ac46762402b426389ab68171593c437)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:36.356692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735bdc6b-9a37-4a53-bc10-3b7e4dc97ca9 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:36.359232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 122/241] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:36 2022] GET /identity/v3/users/1ac46762402b426389ab68171593c437 => generated 297 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:36.364711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735bdc6b-9a37-4a53-bc10-3b7e4dc97ca9 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:36.390964 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9660f-ddff-41c6-af4c-99a41b14804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:36.392142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9660f-ddff-41c6-af4c-99a41b14804d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:36.392627 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9660f-ddff-41c6-af4c-99a41b14804d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:36.393124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9660f-ddff-41c6-af4c-99a41b14804d None admin] PATH_INFO: `/v3/domains/default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:36.394478 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad9660f-ddff-41c6-af4c-99a41b14804d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:36.396132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad9660f-ddff-41c6-af4c-99a41b14804d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:36.404340 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 120/242] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:36.409962 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9660f-ddff-41c6-af4c-99a41b14804d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:36.482731 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735bdc6b-9a37-4a53-bc10-3b7e4dc97ca9 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:36.529226 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528f78a-300f-4855-bc1f-69fc51e60540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:36.530000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528f78a-300f-4855-bc1f-69fc51e60540 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:36.530935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528f78a-300f-4855-bc1f-69fc51e60540 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:36.531219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528f78a-300f-4855-bc1f-69fc51e60540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:36.598428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9660f-ddff-41c6-af4c-99a41b14804d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:36.644680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ef408-ac05-4e2b-82cd-8792184ee798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:36.646192 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ef408-ac05-4e2b-82cd-8792184ee798 None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:36.646674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ef408-ac05-4e2b-82cd-8792184ee798 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:36.647168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ef408-ac05-4e2b-82cd-8792184ee798 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:36.648126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ef408-ac05-4e2b-82cd-8792184ee798 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:36.649768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ef408-ac05-4e2b-82cd-8792184ee798 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:36.663931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528f78a-300f-4855-bc1f-69fc51e60540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:36.663931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528f78a-300f-4855-bc1f-69fc51e60540 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:36.698946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 121/243] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:12:36 2022] POST /identity/v3/users => generated 293 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:12:36.708496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb201e9-974a-4f34-93c5-a1c4a4708bbd None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:36.709149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb201e9-974a-4f34-93c5-a1c4a4708bbd None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:36.709649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb201e9-974a-4f34-93c5-a1c4a4708bbd None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:36.711639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 122/244] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:12:36.718112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6238b7-521b-4a23-90ef-b89b8d657e1c None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:36.718788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6238b7-521b-4a23-90ef-b89b8d657e1c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:36.719262 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6238b7-521b-4a23-90ef-b89b8d657e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:36.740327 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-cb6238b7-521b-4a23-90ef-b89b8d657e1c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:36.741854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-cb6238b7-521b-4a23-90ef-b89b8d657e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:36.786879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 123/245] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 04:12:36.792575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528f78a-300f-4855-bc1f-69fc51e60540 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:36.862800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 123/246] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:36.864204 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be2075-0907-4a93-9fc4-f4b012a7a8be None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:36.864494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be2075-0907-4a93-9fc4-f4b012a7a8be None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:36.864772 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be2075-0907-4a93-9fc4-f4b012a7a8be None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:36.866030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 124/247] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:36.871346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e11c2-352c-4528-af6f-aa7c4f7f07ed None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:36.871639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e11c2-352c-4528-af6f-aa7c4f7f07ed None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:36.871937 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e11c2-352c-4528-af6f-aa7c4f7f07ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:36.901076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-944e11c2-352c-4528-af6f-aa7c4f7f07ed None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:36.902296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-944e11c2-352c-4528-af6f-aa7c4f7f07ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:36.923413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528f78a-300f-4855-bc1f-69fc51e60540 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:36.971406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4351a3c-f105-4e6a-ab69-27780f1761c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:36.972311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4351a3c-f105-4e6a-ab69-27780f1761c7 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:36.972593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4351a3c-f105-4e6a-ab69-27780f1761c7 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:36.972865 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4351a3c-f105-4e6a-ab69-27780f1761c7 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:36.974758 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4351a3c-f105-4e6a-ab69-27780f1761c7 None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:36.975625 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4351a3c-f105-4e6a-ab69-27780f1761c7 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:36.977689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 124/248] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:36 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:36.978463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4351a3c-f105-4e6a-ab69-27780f1761c7 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:36.987685 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 125/249] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:36.988520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e11c2-352c-4528-af6f-aa7c4f7f07ed None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:37.097056 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4351a3c-f105-4e6a-ab69-27780f1761c7 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:37.115860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e11c2-352c-4528-af6f-aa7c4f7f07ed None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:37.145070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d2e6f-2d02-41f4-8cc7-ea1b4af4cb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:37.145899 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d2e6f-2d02-41f4-8cc7-ea1b4af4cb4f None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:37.146425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d2e6f-2d02-41f4-8cc7-ea1b4af4cb4f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:37.146425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d2e6f-2d02-41f4-8cc7-ea1b4af4cb4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:37.159342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad37cf-2740-4f1c-82d0-e135ca2e71a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:37.159785 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d2e6f-2d02-41f4-8cc7-ea1b4af4cb4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:37.160076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad37cf-2740-4f1c-82d0-e135ca2e71a2 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:37.160365 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad37cf-2740-4f1c-82d0-e135ca2e71a2 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:37.160640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad37cf-2740-4f1c-82d0-e135ca2e71a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:37.161478 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad37cf-2740-4f1c-82d0-e135ca2e71a2 None admin] RBAC: Adding query filter params (role.id=2a69f0e80033468f96c13ef5f64d094d, scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f, user.id=1ac46762402b426389ab68171593c437) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:37.161789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad37cf-2740-4f1c-82d0-e135ca2e71a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:37.163366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad37cf-2740-4f1c-82d0-e135ca2e71a2 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-973d2e6f-2d02-41f4-8cc7-ea1b4af4cb4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:37.168819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:37.169732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:37.171523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:37.171523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:37.171523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:37.171523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:37.171523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:37.171523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:37.171523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:37.171523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:37.171523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:37.171523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:37.171523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:37.171523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:37.171523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 125/250] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:37.172574 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 126/251] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:36 2022] GET /identity/v3/role_assignments?user.id=1ac46762402b426389ab68171593c437&scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f&role.id=2a69f0e80033468f96c13ef5f64d094d => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:37.172648 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d2e6f-2d02-41f4-8cc7-ea1b4af4cb4f None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:37.177226 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad37cf-2740-4f1c-82d0-e135ca2e71a2 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:37.303131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d2e6f-2d02-41f4-8cc7-ea1b4af4cb4f None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:37.318192 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad37cf-2740-4f1c-82d0-e135ca2e71a2 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:37.338552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb33823-5f7e-4df8-a982-cf92b6c5d331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:37.339369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb33823-5f7e-4df8-a982-cf92b6c5d331 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:37.339651 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb33823-5f7e-4df8-a982-cf92b6c5d331 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:37.339947 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb33823-5f7e-4df8-a982-cf92b6c5d331 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:37.350993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb33823-5f7e-4df8-a982-cf92b6c5d331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-efb33823-5f7e-4df8-a982-cf92b6c5d331 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:37.363932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:37.364884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:37.365665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:37.365665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:37.365665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:37.365665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:37.365665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:37.365665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:37.365665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:37.365665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:37.365665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:37.365665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:37.365665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:37.365665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:37.365665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 126/252] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:12:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:37.366803 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb33823-5f7e-4df8-a982-cf92b6c5d331 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:37.370771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbb976-fc41-46aa-b776-cea88eb93f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:37.372694 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbb976-fc41-46aa-b776-cea88eb93f12 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:37.373209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbb976-fc41-46aa-b776-cea88eb93f12 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:37.373752 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbb976-fc41-46aa-b776-cea88eb93f12 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:37.375233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dbb976-fc41-46aa-b776-cea88eb93f12 None admin] RBAC: Adding query filter params (name=member) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:37.375767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dbb976-fc41-46aa-b776-cea88eb93f12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:37.377428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dbb976-fc41-46aa-b776-cea88eb93f12 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:37.388021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 127/253] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:37.395203 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbb976-fc41-46aa-b776-cea88eb93f12 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:37.522028 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbb976-fc41-46aa-b776-cea88eb93f12 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:37.522378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb33823-5f7e-4df8-a982-cf92b6c5d331 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:37.567941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2f838-4d94-4bc3-a647-000edd520ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:37.569200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2f838-4d94-4bc3-a647-000edd520ccc None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:37.570681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2f838-4d94-4bc3-a647-000edd520ccc None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:37.571161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2f838-4d94-4bc3-a647-000edd520ccc None admin] PATH_INFO: `/v3/users/b857f9f672f1497cad7c9e114cb66e80` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:37.574568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2f838-4d94-4bc3-a647-000edd520ccc None admin] RBAC: Authorizing `identity:get_user(user_id=b857f9f672f1497cad7c9e114cb66e80)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:37.578731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2f838-4d94-4bc3-a647-000edd520ccc None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:37.583223 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 128/254] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:37 2022] GET /identity/v3/users/b857f9f672f1497cad7c9e114cb66e80 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:37.590158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2f838-4d94-4bc3-a647-000edd520ccc None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:37.602749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a3aae-a47f-478d-a33e-9c3cc07f9c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:37.603792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a3aae-a47f-478d-a33e-9c3cc07f9c0a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:37.604495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a3aae-a47f-478d-a33e-9c3cc07f9c0a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:37.604987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a3aae-a47f-478d-a33e-9c3cc07f9c0a None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:37.605945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a3aae-a47f-478d-a33e-9c3cc07f9c0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:37.606447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a3aae-a47f-478d-a33e-9c3cc07f9c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:37.608680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a3aae-a47f-478d-a33e-9c3cc07f9c0a None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:37.627779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 127/255] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:37.633927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a3aae-a47f-478d-a33e-9c3cc07f9c0a None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:37.800391 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2f838-4d94-4bc3-a647-000edd520ccc None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:37.810669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a3aae-a47f-478d-a33e-9c3cc07f9c0a None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:37.845958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eabee-fe6e-44dc-9925-578599fbb847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:37.847224 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eabee-fe6e-44dc-9925-578599fbb847 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:37.847708 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eabee-fe6e-44dc-9925-578599fbb847 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:37.848345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eabee-fe6e-44dc-9925-578599fbb847 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:37.852875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808eabee-fe6e-44dc-9925-578599fbb847 None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:37.854917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808eabee-fe6e-44dc-9925-578599fbb847 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:37.857181 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 129/256] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:37 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:37.863179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eabee-fe6e-44dc-9925-578599fbb847 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:37.863969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be4810-7f07-46ec-84ee-7c9892a14517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:37.864879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be4810-7f07-46ec-84ee-7c9892a14517 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:37.865162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be4810-7f07-46ec-84ee-7c9892a14517 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:37.865438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be4810-7f07-46ec-84ee-7c9892a14517 None admin] PATH_INFO: `/v3/users/9014bf749deb446091ec762123c0d404` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:37.866654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be4810-7f07-46ec-84ee-7c9892a14517 None admin] RBAC: Authorizing `identity:get_user(user_id=9014bf749deb446091ec762123c0d404)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:37.868158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be4810-7f07-46ec-84ee-7c9892a14517 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:37.869954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 128/257] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:37 2022] GET /identity/v3/users/9014bf749deb446091ec762123c0d404 => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:37.883943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be4810-7f07-46ec-84ee-7c9892a14517 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:38.091238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be4810-7f07-46ec-84ee-7c9892a14517 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:38.110114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eabee-fe6e-44dc-9925-578599fbb847 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:38.171680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4aa36-c8d5-43cd-913f-cff4e4932cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:38.172373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4aa36-c8d5-43cd-913f-cff4e4932cde None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:38.172658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4aa36-c8d5-43cd-913f-cff4e4932cde None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:38.172934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4aa36-c8d5-43cd-913f-cff4e4932cde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:38.192656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f1441-acf0-4ca0-adbd-5e398ea0184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:38.193565 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f1441-acf0-4ca0-adbd-5e398ea0184f None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:38.193846 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f1441-acf0-4ca0-adbd-5e398ea0184f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:38.194120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f1441-acf0-4ca0-adbd-5e398ea0184f None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f/users/b857f9f672f1497cad7c9e114cb66e80/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:38.198799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f1441-acf0-4ca0-adbd-5e398ea0184f None admin] RBAC: Authorizing `identity:create_grant(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f, user_id=b857f9f672f1497cad7c9e114cb66e80, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:38.200289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f1441-acf0-4ca0-adbd-5e398ea0184f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:38.230765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 130/258] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:12:37 2022] PUT /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f/users/b857f9f672f1497cad7c9e114cb66e80/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:12:38.232281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7daffd-6b2c-42e9-bfa6-1de098737cc9 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:38.232577 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7daffd-6b2c-42e9-bfa6-1de098737cc9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:38.232861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7daffd-6b2c-42e9-bfa6-1de098737cc9 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:38.234302 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 131/259] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:38.244960 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57882fa-f5f4-4f05-8c2a-edae2be1e8bf None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:38.245319 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57882fa-f5f4-4f05-8c2a-edae2be1e8bf None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:38.245528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57882fa-f5f4-4f05-8c2a-edae2be1e8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:38.277839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d57882fa-f5f4-4f05-8c2a-edae2be1e8bf None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:38.279074 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d57882fa-f5f4-4f05-8c2a-edae2be1e8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:38.309274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4aa36-c8d5-43cd-913f-cff4e4932cde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:38.310760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4aa36-c8d5-43cd-913f-cff4e4932cde None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:38.413836 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 132/260] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:38.448854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57882fa-f5f4-4f05-8c2a-edae2be1e8bf None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:38.496284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 129/261] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:38.499405 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4aa36-c8d5-43cd-913f-cff4e4932cde None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:38.619718 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4aa36-c8d5-43cd-913f-cff4e4932cde None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:38.644372 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57882fa-f5f4-4f05-8c2a-edae2be1e8bf None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:38.656458 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981d4a5-0578-4d6a-a18b-6751172d13c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:38.657337 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981d4a5-0578-4d6a-a18b-6751172d13c7 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:38.657618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981d4a5-0578-4d6a-a18b-6751172d13c7 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:38.657893 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981d4a5-0578-4d6a-a18b-6751172d13c7 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:38.659094 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981d4a5-0578-4d6a-a18b-6751172d13c7 None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:38.660593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981d4a5-0578-4d6a-a18b-6751172d13c7 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:38.662306 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 130/262] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:38 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:38.668208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981d4a5-0578-4d6a-a18b-6751172d13c7 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:38.692614 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe59ffe-baa0-4cbe-8b70-8c7930e79328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:38.693455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe59ffe-baa0-4cbe-8b70-8c7930e79328 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:38.693734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe59ffe-baa0-4cbe-8b70-8c7930e79328 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:38.694068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe59ffe-baa0-4cbe-8b70-8c7930e79328 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:38.705473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe59ffe-baa0-4cbe-8b70-8c7930e79328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-efe59ffe-baa0-4cbe-8b70-8c7930e79328 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:38.713737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:38.714619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:38.715405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:38.715405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:38.715405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:38.715405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:38.715405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:38.715405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:38.715405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:38.715405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:38.715405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:38.715405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:38.715405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:38.715405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:38.715405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 133/263] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:12:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:12:38.720919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe59ffe-baa0-4cbe-8b70-8c7930e79328 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:38.798218 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981d4a5-0578-4d6a-a18b-6751172d13c7 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:38.846514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae758a3-5eb8-453f-976b-759b8679d7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:38.847355 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae758a3-5eb8-453f-976b-759b8679d7b1 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:38.848083 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae758a3-5eb8-453f-976b-759b8679d7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:38.848369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae758a3-5eb8-453f-976b-759b8679d7b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:38.849522 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae758a3-5eb8-453f-976b-759b8679d7b1 None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f, user.id=9014bf749deb446091ec762123c0d404) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:38.849841 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae758a3-5eb8-453f-976b-759b8679d7b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:38.851797 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae758a3-5eb8-453f-976b-759b8679d7b1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:38.860928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 131/264] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:38 2022] GET /identity/v3/role_assignments?user.id=9014bf749deb446091ec762123c0d404&scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f&role.id=e59113c0366049e889df3f55eed90c72 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:38.875844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe59ffe-baa0-4cbe-8b70-8c7930e79328 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:38.932748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a162d77-1206-43e9-82f7-87ff685ee631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:38.933359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a162d77-1206-43e9-82f7-87ff685ee631 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:38.933646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a162d77-1206-43e9-82f7-87ff685ee631 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:38.933923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a162d77-1206-43e9-82f7-87ff685ee631 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:38.934685 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a162d77-1206-43e9-82f7-87ff685ee631 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:38.934997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a162d77-1206-43e9-82f7-87ff685ee631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:38.936695 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a162d77-1206-43e9-82f7-87ff685ee631 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:38.945720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 134/265] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:12:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:38.950926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae758a3-5eb8-453f-976b-759b8679d7b1 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:39.076101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae758a3-5eb8-453f-976b-759b8679d7b1 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:39.118671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a6aeb-eca2-4d46-bd0d-82cb6354e8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:39.119564 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a6aeb-eca2-4d46-bd0d-82cb6354e8c7 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:39.119847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a6aeb-eca2-4d46-bd0d-82cb6354e8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:39.120150 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a6aeb-eca2-4d46-bd0d-82cb6354e8c7 None admin] PATH_INFO: `/v3/users/9014bf749deb446091ec762123c0d404` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:39.121377 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a6aeb-eca2-4d46-bd0d-82cb6354e8c7 None admin] RBAC: Authorizing `identity:get_user(user_id=9014bf749deb446091ec762123c0d404)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:39.122872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a6aeb-eca2-4d46-bd0d-82cb6354e8c7 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:39.125972 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 132/266] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:38 2022] GET /identity/v3/users/9014bf749deb446091ec762123c0d404 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:39.137096 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a162d77-1206-43e9-82f7-87ff685ee631 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:39.282844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a162d77-1206-43e9-82f7-87ff685ee631 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:39.325908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcbf28a-3071-45dc-964b-bdf9e9d3b846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:39.326567 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbf28a-3071-45dc-964b-bdf9e9d3b846 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:39.327449 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbf28a-3071-45dc-964b-bdf9e9d3b846 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:39.327730 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbf28a-3071-45dc-964b-bdf9e9d3b846 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:39.408129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcbf28a-3071-45dc-964b-bdf9e9d3b846 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:39.409613 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcbf28a-3071-45dc-964b-bdf9e9d3b846 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:39.529398 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 135/267] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:39.536026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a6aeb-eca2-4d46-bd0d-82cb6354e8c7 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:39.646105 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a6aeb-eca2-4d46-bd0d-82cb6354e8c7 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:39.685944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd306a-4447-4b35-9822-5465be916247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:39.687092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd306a-4447-4b35-9822-5465be916247 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:39.687448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd306a-4447-4b35-9822-5465be916247 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:39.687739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd306a-4447-4b35-9822-5465be916247 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:39.688995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd306a-4447-4b35-9822-5465be916247 None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:39.690469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd306a-4447-4b35-9822-5465be916247 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:39.692393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 133/268] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:39 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:39.698151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcbf28a-3071-45dc-964b-bdf9e9d3b846 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:39.810633 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcbf28a-3071-45dc-964b-bdf9e9d3b846 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:39.855473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04007a7-4562-48d2-b463-8bfa6da851ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:39.856176 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04007a7-4562-48d2-b463-8bfa6da851ab None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:39.856456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04007a7-4562-48d2-b463-8bfa6da851ab None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:39.856729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04007a7-4562-48d2-b463-8bfa6da851ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:39.857568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04007a7-4562-48d2-b463-8bfa6da851ab None admin] RBAC: Adding query filter params (role.id=7600100dce614d9bb4d47b1e660b7bb2, scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f, user.id=9014bf749deb446091ec762123c0d404) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:39.857878 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04007a7-4562-48d2-b463-8bfa6da851ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:39.859339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04007a7-4562-48d2-b463-8bfa6da851ab None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:39.868451 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 136/269] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:39 2022] GET /identity/v3/role_assignments?user.id=9014bf749deb446091ec762123c0d404&scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f&role.id=7600100dce614d9bb4d47b1e660b7bb2 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:40.193737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2283c9-50eb-47d2-b70d-3831aaf5f766 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:40.194723 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2283c9-50eb-47d2-b70d-3831aaf5f766 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:40.195211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2283c9-50eb-47d2-b70d-3831aaf5f766 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:40.196934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 134/270] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:40.203037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ade86-09d4-40af-a12e-10428f72079d None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:40.203728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ade86-09d4-40af-a12e-10428f72079d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:40.203872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ade86-09d4-40af-a12e-10428f72079d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:40.223519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-1a9ade86-09d4-40af-a12e-10428f72079d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:40.224950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1a9ade86-09d4-40af-a12e-10428f72079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:40.303551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 137/271] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:40.324614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37a4a7-606c-400f-a56b-9cd0c764e3ad None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:40.325220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37a4a7-606c-400f-a56b-9cd0c764e3ad None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:40.325702 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37a4a7-606c-400f-a56b-9cd0c764e3ad None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:40.327966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 135/272] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:40.333303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ade86-09d4-40af-a12e-10428f72079d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:40.333805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ee728-2239-4b98-8450-c941248b30f7 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:40.334293 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ee728-2239-4b98-8450-c941248b30f7 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:40.334770 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ee728-2239-4b98-8450-c941248b30f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:40.359912 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-5f2ee728-2239-4b98-8450-c941248b30f7 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:40.361462 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5f2ee728-2239-4b98-8450-c941248b30f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:40.431644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ade86-09d4-40af-a12e-10428f72079d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:40.441100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 136/273] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:40.466207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4a91f-123f-4df2-9fa8-e5bf53ccd5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:40.466910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a91f-123f-4df2-9fa8-e5bf53ccd5ba None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:40.467179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a91f-123f-4df2-9fa8-e5bf53ccd5ba None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:40.468240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a91f-123f-4df2-9fa8-e5bf53ccd5ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:40.470877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ee728-2239-4b98-8450-c941248b30f7 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:40.479447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4a91f-123f-4df2-9fa8-e5bf53ccd5ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-34e4a91f-123f-4df2-9fa8-e5bf53ccd5ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:40.487138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:40.488067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:40.489182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:40.489182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:40.489182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:40.489182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:40.489182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:40.489182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:40.489182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:40.489182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:40.489182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:40.489182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:40.489182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:40.489182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:40.489182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 138/274] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:40.494655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2e2b8-97b5-4841-b04f-df307cade6e5 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:40.495494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2e2b8-97b5-4841-b04f-df307cade6e5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:40.496273 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2e2b8-97b5-4841-b04f-df307cade6e5 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:40.502889 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 139/275] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:40 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:40.502889 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2e2b8-97b5-4841-b04f-df307cade6e5 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:40.569652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ee728-2239-4b98-8450-c941248b30f7 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:40.601891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2e2b8-97b5-4841-b04f-df307cade6e5 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:40.605347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86604c-d87b-4ec9-ad4f-7886922e276d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:40.606155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86604c-d87b-4ec9-ad4f-7886922e276d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:40.606439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86604c-d87b-4ec9-ad4f-7886922e276d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:40.606718 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86604c-d87b-4ec9-ad4f-7886922e276d None admin] PATH_INFO: `/v3/domains/default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:40.607885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86604c-d87b-4ec9-ad4f-7886922e276d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:40.610377 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86604c-d87b-4ec9-ad4f-7886922e276d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:40.612880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 137/276] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:40.614044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284413b7-8e0e-4f36-a373-35eb8a71895c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:40.614337 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284413b7-8e0e-4f36-a373-35eb8a71895c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:40.614623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284413b7-8e0e-4f36-a373-35eb8a71895c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:40.631405 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-284413b7-8e0e-4f36-a373-35eb8a71895c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:40.632602 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-284413b7-8e0e-4f36-a373-35eb8a71895c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:40.636270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a8557-c6e5-46e4-898c-cd2b2c195163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:40.637161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a8557-c6e5-46e4-898c-cd2b2c195163 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:40.637635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a8557-c6e5-46e4-898c-cd2b2c195163 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:40.638125 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a8557-c6e5-46e4-898c-cd2b2c195163 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:40.639135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a8557-c6e5-46e4-898c-cd2b2c195163 None admin] RBAC: Adding query filter params (name=member) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:40.639674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a8557-c6e5-46e4-898c-cd2b2c195163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:40.641189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a8557-c6e5-46e4-898c-cd2b2c195163 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:40.650477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 140/277] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:40.651279 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a8557-c6e5-46e4-898c-cd2b2c195163 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:40.697369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 138/278] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:40.698420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284413b7-8e0e-4f36-a373-35eb8a71895c None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:40.765079 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a8557-c6e5-46e4-898c-cd2b2c195163 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:40.798315 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e4060-8235-4432-b93b-aa5d80ba2dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:40.798983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e4060-8235-4432-b93b-aa5d80ba2dee None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:40.799261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e4060-8235-4432-b93b-aa5d80ba2dee None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:40.799532 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e4060-8235-4432-b93b-aa5d80ba2dee None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:40.799996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284413b7-8e0e-4f36-a373-35eb8a71895c None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:40.800271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e4060-8235-4432-b93b-aa5d80ba2dee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:40.801701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e4060-8235-4432-b93b-aa5d80ba2dee None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:40.823478 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 141/279] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:12:40 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:12:40.833098 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e4060-8235-4432-b93b-aa5d80ba2dee None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:40.837747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8575cb0-45b5-48d5-8b1f-6aacd7c3c40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:40.839114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8575cb0-45b5-48d5-8b1f-6aacd7c3c40f None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:40.839632 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8575cb0-45b5-48d5-8b1f-6aacd7c3c40f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:40.840149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8575cb0-45b5-48d5-8b1f-6aacd7c3c40f None admin] PATH_INFO: `/v3/users/cb3cff94a7c742d8a79e6c5c4244dad9` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:40.857709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8575cb0-45b5-48d5-8b1f-6aacd7c3c40f None admin] RBAC: Authorizing `identity:get_user(user_id=cb3cff94a7c742d8a79e6c5c4244dad9)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:40.860294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8575cb0-45b5-48d5-8b1f-6aacd7c3c40f None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:40.867421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 139/280] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:40 2022] GET /identity/v3/users/cb3cff94a7c742d8a79e6c5c4244dad9 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:40.869089 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8575cb0-45b5-48d5-8b1f-6aacd7c3c40f None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:40.945705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e4060-8235-4432-b93b-aa5d80ba2dee None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:40.982651 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e52491-fc38-4c73-baca-e9674d1990f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:40.983111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8575cb0-45b5-48d5-8b1f-6aacd7c3c40f None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:40.983786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e52491-fc38-4c73-baca-e9674d1990f5 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:40.983786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e52491-fc38-4c73-baca-e9674d1990f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:40.984086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e52491-fc38-4c73-baca-e9674d1990f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:40.995885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e52491-fc38-4c73-baca-e9674d1990f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-76e52491-fc38-4c73-baca-e9674d1990f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:41.004047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:41.005017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:41.006231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:41.006231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:41.006231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:41.006231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:41.006231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:41.006231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:41.006231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:41.006231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:41.006231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:41.006231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:41.006231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:41.006231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:41.006231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 142/281] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:41.011495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e52491-fc38-4c73-baca-e9674d1990f5 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:41.029091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb5fbd-c65e-4c60-8cdd-085a6d567688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:41.029771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb5fbd-c65e-4c60-8cdd-085a6d567688 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:41.030661 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb5fbd-c65e-4c60-8cdd-085a6d567688 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:41.031648 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb5fbd-c65e-4c60-8cdd-085a6d567688 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:41.032587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb5fbd-c65e-4c60-8cdd-085a6d567688 None admin] RBAC: Adding query filter params (role.id=d8b94102152a46768d568f66794854f3, user.id=cb3cff94a7c742d8a79e6c5c4244dad9, scope.system=all) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:41.033172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb5fbd-c65e-4c60-8cdd-085a6d567688 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:41.034373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb5fbd-c65e-4c60-8cdd-085a6d567688 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:41.040110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 140/282] 10.222.0.44 () {56 vars in 1308 bytes} [Mon May 9 04:12:40 2022] GET /identity/v3/role_assignments?user.id=cb3cff94a7c742d8a79e6c5c4244dad9&scope.system=all&role.id=d8b94102152a46768d568f66794854f3 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:41.126925 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e52491-fc38-4c73-baca-e9674d1990f5 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:41.164571 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a91f80-ba28-47a1-a2f6-f6d1696d9053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:41.165271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a91f80-ba28-47a1-a2f6-f6d1696d9053 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:41.165556 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a91f80-ba28-47a1-a2f6-f6d1696d9053 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:41.165835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a91f80-ba28-47a1-a2f6-f6d1696d9053 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:41.166604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a91f80-ba28-47a1-a2f6-f6d1696d9053 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:41.166912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a91f80-ba28-47a1-a2f6-f6d1696d9053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:41.168357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a91f80-ba28-47a1-a2f6-f6d1696d9053 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:41.178806 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 143/283] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:41.184706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb5fbd-c65e-4c60-8cdd-085a6d567688 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:41.220963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4b7eb-1f35-4580-b08f-40aa91d26d6f None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:41.221477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4b7eb-1f35-4580-b08f-40aa91d26d6f None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:41.221549 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4b7eb-1f35-4580-b08f-40aa91d26d6f None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:41.223583 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 144/284] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:12:41.229468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76746b3-d346-4f37-95e7-f7b4a2674685 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:41.229780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76746b3-d346-4f37-95e7-f7b4a2674685 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:41.230032 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76746b3-d346-4f37-95e7-f7b4a2674685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:41.251627 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-e76746b3-d346-4f37-95e7-f7b4a2674685 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:41.252858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e76746b3-d346-4f37-95e7-f7b4a2674685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:41.313229 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb5fbd-c65e-4c60-8cdd-085a6d567688 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:41.331091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 145/285] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:41.356750 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76746b3-d346-4f37-95e7-f7b4a2674685 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:41.359068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09fd67-3f4b-4866-b6fe-ff6674cbad8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:41.360265 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09fd67-3f4b-4866-b6fe-ff6674cbad8d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:41.360549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09fd67-3f4b-4866-b6fe-ff6674cbad8d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:41.360818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09fd67-3f4b-4866-b6fe-ff6674cbad8d None admin] PATH_INFO: `/v3/users/1ac46762402b426389ab68171593c437` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:41.362907 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09fd67-3f4b-4866-b6fe-ff6674cbad8d None admin] RBAC: Authorizing `identity:get_user(user_id=1ac46762402b426389ab68171593c437)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:41.364464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09fd67-3f4b-4866-b6fe-ff6674cbad8d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:41.366108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 141/286] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:41 2022] GET /identity/v3/users/1ac46762402b426389ab68171593c437 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:41.371921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09fd67-3f4b-4866-b6fe-ff6674cbad8d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:41.459817 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76746b3-d346-4f37-95e7-f7b4a2674685 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:41.472482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09fd67-3f4b-4866-b6fe-ff6674cbad8d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:41.495271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cb8d8-885b-4f10-b447-df5faeffdc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:41.496475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cb8d8-885b-4f10-b447-df5faeffdc23 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:41.497042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cb8d8-885b-4f10-b447-df5faeffdc23 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:41.497526 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cb8d8-885b-4f10-b447-df5faeffdc23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:41.508242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469cb8d8-885b-4f10-b447-df5faeffdc23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:41.515711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01378d-464d-4fbe-94e2-adeb156f2fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:41.516613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01378d-464d-4fbe-94e2-adeb156f2fb7 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:41.516895 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01378d-464d-4fbe-94e2-adeb156f2fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:41.517173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01378d-464d-4fbe-94e2-adeb156f2fb7 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-469cb8d8-885b-4f10-b447-df5faeffdc23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:41.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:41.518431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:41.519225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:41.519225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:41.519225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:41.519225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:41.519225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:41.519225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:41.519225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:41.519225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:41.519225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:41.519225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:41.519225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:41.519225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:41.521866 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 146/287] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:12:41.526506 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01378d-464d-4fbe-94e2-adeb156f2fb7 None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:41.526506 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc01378d-464d-4fbe-94e2-adeb156f2fb7 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:41.527729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cb8d8-885b-4f10-b447-df5faeffdc23 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:41.528129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 142/288] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:41 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:41.534330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01378d-464d-4fbe-94e2-adeb156f2fb7 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:41.631393 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cb8d8-885b-4f10-b447-df5faeffdc23 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:41.666262 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c365e71-3d70-4aa8-86cf-7e464b47d973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:41.667323 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c365e71-3d70-4aa8-86cf-7e464b47d973 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:41.667804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c365e71-3d70-4aa8-86cf-7e464b47d973 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:41.668303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c365e71-3d70-4aa8-86cf-7e464b47d973 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:41.669267 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c365e71-3d70-4aa8-86cf-7e464b47d973 None admin] RBAC: Adding query filter params (name=member) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:41.669790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c365e71-3d70-4aa8-86cf-7e464b47d973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:41.671390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c365e71-3d70-4aa8-86cf-7e464b47d973 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:41.680012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 147/289] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:41.685899 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c365e71-3d70-4aa8-86cf-7e464b47d973 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:41.737439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01378d-464d-4fbe-94e2-adeb156f2fb7 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:41.799092 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c365e71-3d70-4aa8-86cf-7e464b47d973 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:41.817059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b1e5f-9687-47b3-a7b4-bcaf8933fde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:41.818079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b1e5f-9687-47b3-a7b4-bcaf8933fde0 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:41.818368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b1e5f-9687-47b3-a7b4-bcaf8933fde0 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:41.818646 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b1e5f-9687-47b3-a7b4-bcaf8933fde0 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f/users/1ac46762402b426389ab68171593c437/roles/2a69f0e80033468f96c13ef5f64d094d` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:41.820664 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b1e5f-9687-47b3-a7b4-bcaf8933fde0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f, user_id=1ac46762402b426389ab68171593c437, role_id=2a69f0e80033468f96c13ef5f64d094d)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:41.822108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b1e5f-9687-47b3-a7b4-bcaf8933fde0 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:41.835621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9e5df-31bf-40ba-92ef-ce40c0a78b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:41.836573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9e5df-31bf-40ba-92ef-ce40c0a78b8f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:41.836967 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9e5df-31bf-40ba-92ef-ce40c0a78b8f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:41.837278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9e5df-31bf-40ba-92ef-ce40c0a78b8f None admin] PATH_INFO: `/v3/users/b857f9f672f1497cad7c9e114cb66e80` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:41.838550 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9e5df-31bf-40ba-92ef-ce40c0a78b8f None admin] RBAC: Authorizing `identity:get_user(user_id=b857f9f672f1497cad7c9e114cb66e80)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:41.840093 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9e5df-31bf-40ba-92ef-ce40c0a78b8f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:41.842896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 148/290] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:41 2022] GET /identity/v3/users/b857f9f672f1497cad7c9e114cb66e80 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:41.848517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9e5df-31bf-40ba-92ef-ce40c0a78b8f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:41.853350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 143/291] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:12:41 2022] PUT /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f/users/1ac46762402b426389ab68171593c437/roles/2a69f0e80033468f96c13ef5f64d094d => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:12:41.959528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9e5df-31bf-40ba-92ef-ce40c0a78b8f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:41.994933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e297102-7154-48c8-a7f0-3fb60a6c6d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:41.995603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e297102-7154-48c8-a7f0-3fb60a6c6d68 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:41.995883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e297102-7154-48c8-a7f0-3fb60a6c6d68 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:41.996186 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e297102-7154-48c8-a7f0-3fb60a6c6d68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:42.071987 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e297102-7154-48c8-a7f0-3fb60a6c6d68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:42.073476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e297102-7154-48c8-a7f0-3fb60a6c6d68 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:42.200125 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 149/292] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:42.206383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b1e5f-9687-47b3-a7b4-bcaf8933fde0 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:42.309886 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b1e5f-9687-47b3-a7b4-bcaf8933fde0 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:42.364794 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320083eb-82bf-430f-bf8e-b6696653ca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:42.365663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320083eb-82bf-430f-bf8e-b6696653ca48 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:42.365946 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320083eb-82bf-430f-bf8e-b6696653ca48 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:42.366223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320083eb-82bf-430f-bf8e-b6696653ca48 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:42.372588 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320083eb-82bf-430f-bf8e-b6696653ca48 None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:42.374840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320083eb-82bf-430f-bf8e-b6696653ca48 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:42.376642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 144/293] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:42 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:42.382487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e297102-7154-48c8-a7f0-3fb60a6c6d68 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:42.510731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e297102-7154-48c8-a7f0-3fb60a6c6d68 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:42.557883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a26ca-e705-4ec1-9389-d770bd97967b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:42.558545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a26ca-e705-4ec1-9389-d770bd97967b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:42.558820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a26ca-e705-4ec1-9389-d770bd97967b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:42.559089 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a26ca-e705-4ec1-9389-d770bd97967b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:42.560049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a26ca-e705-4ec1-9389-d770bd97967b None admin] RBAC: Adding query filter params (role.id=d8b94102152a46768d568f66794854f3, scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f, user.id=b857f9f672f1497cad7c9e114cb66e80) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:42.560309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a26ca-e705-4ec1-9389-d770bd97967b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:42.562342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a26ca-e705-4ec1-9389-d770bd97967b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:42.572535 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 150/294] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:42 2022] GET /identity/v3/role_assignments?user.id=b857f9f672f1497cad7c9e114cb66e80&scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f&role.id=d8b94102152a46768d568f66794854f3 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:43.584542 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc350a62-02bb-4427-b478-9cd2141ba23d None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:43.585235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc350a62-02bb-4427-b478-9cd2141ba23d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:43.585235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc350a62-02bb-4427-b478-9cd2141ba23d None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:43.586573 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 145/295] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:43.595119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b7be2-e65c-42ce-9f81-5e3d7c1bbe1a None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:43.595119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b7be2-e65c-42ce-9f81-5e3d7c1bbe1a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:43.595119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b7be2-e65c-42ce-9f81-5e3d7c1bbe1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:43.613580 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d70b7be2-e65c-42ce-9f81-5e3d7c1bbe1a None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:43.615086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d70b7be2-e65c-42ce-9f81-5e3d7c1bbe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:43.628406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8489ad-9776-4149-97a3-15e5adba9d52 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:43.628693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8489ad-9776-4149-97a3-15e5adba9d52 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:43.628972 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8489ad-9776-4149-97a3-15e5adba9d52 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:43.630584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 146/296] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:43.635541 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c4204-b2d0-4fad-a15b-4a915c1ada2e None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:43.635828 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c4204-b2d0-4fad-a15b-4a915c1ada2e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:43.636130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c4204-b2d0-4fad-a15b-4a915c1ada2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:43.653057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-936c4204-b2d0-4fad-a15b-4a915c1ada2e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:43.653943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-936c4204-b2d0-4fad-a15b-4a915c1ada2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:43.685233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 151/297] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:43.710489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b7be2-e65c-42ce-9f81-5e3d7c1bbe1a None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:43.732948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 147/298] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:43.766641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c4204-b2d0-4fad-a15b-4a915c1ada2e None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:43.837572 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b7be2-e65c-42ce-9f81-5e3d7c1bbe1a None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:43.879045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac241314-7191-4c80-ade8-adf0ede14836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:43.880820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac241314-7191-4c80-ade8-adf0ede14836 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:43.881559 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac241314-7191-4c80-ade8-adf0ede14836 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:43.882035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac241314-7191-4c80-ade8-adf0ede14836 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:43.883713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c4204-b2d0-4fad-a15b-4a915c1ada2e None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:43.884225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac241314-7191-4c80-ade8-adf0ede14836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:43.886451 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac241314-7191-4c80-ade8-adf0ede14836 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:43.888390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 152/299] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:43.897465 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac241314-7191-4c80-ade8-adf0ede14836 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:43.926819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba0eef-b10d-42c8-a9b0-9933184eb654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:43.927650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba0eef-b10d-42c8-a9b0-9933184eb654 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:43.927960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba0eef-b10d-42c8-a9b0-9933184eb654 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:43.928266 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba0eef-b10d-42c8-a9b0-9933184eb654 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:43.940451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba0eef-b10d-42c8-a9b0-9933184eb654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-49ba0eef-b10d-42c8-a9b0-9933184eb654 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:43.953010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:43.954213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:43.955131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:43.955131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:43.955131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:43.955131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:43.955131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:43.955131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:43.955131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:43.955131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:43.955131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:43.955131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:43.955131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:43.955131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:43.957742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 148/300] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:43.959327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d497040-366b-48b3-9637-cf206559f294 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:43.959838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d497040-366b-48b3-9637-cf206559f294 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:43.960393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d497040-366b-48b3-9637-cf206559f294 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:43.963940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 149/301] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:43.965090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d497040-366b-48b3-9637-cf206559f294 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:44.017556 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac241314-7191-4c80-ade8-adf0ede14836 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:44.056355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a1cca-18b0-404d-aa12-b6badfcf4d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:44.057027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a1cca-18b0-404d-aa12-b6badfcf4d8e None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:44.057305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a1cca-18b0-404d-aa12-b6badfcf4d8e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:44.057635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a1cca-18b0-404d-aa12-b6badfcf4d8e None admin] PATH_INFO: `/v3/groups` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:44.058265 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a1cca-18b0-404d-aa12-b6badfcf4d8e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:44.059705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a1cca-18b0-404d-aa12-b6badfcf4d8e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:44.083929 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 153/302] 10.222.0.44 () {60 vars in 1134 bytes} [Mon May 9 04:12:43 2022] POST /identity/v3/groups => generated 229 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:12:44.083929 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c143e3-c1b3-48fd-83b1-bcdb11f078bf None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:44.083929 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c143e3-c1b3-48fd-83b1-bcdb11f078bf None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:44.083929 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c143e3-c1b3-48fd-83b1-bcdb11f078bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:44.094511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d497040-366b-48b3-9637-cf206559f294 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:44.113147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-93c143e3-c1b3-48fd-83b1-bcdb11f078bf None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:44.115239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-93c143e3-c1b3-48fd-83b1-bcdb11f078bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:44.144920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b20454-99a1-468b-b5c6-a58e0dc9c321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:44.145624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b20454-99a1-468b-b5c6-a58e0dc9c321 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:44.145960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b20454-99a1-468b-b5c6-a58e0dc9c321 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:44.146274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b20454-99a1-468b-b5c6-a58e0dc9c321 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:44.147685 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b20454-99a1-468b-b5c6-a58e0dc9c321 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:44.148008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b20454-99a1-468b-b5c6-a58e0dc9c321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:44.149983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b20454-99a1-468b-b5c6-a58e0dc9c321 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:44.164308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 150/303] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:44.171704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b20454-99a1-468b-b5c6-a58e0dc9c321 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:44.218414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 154/304] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:44.243545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c143e3-c1b3-48fd-83b1-bcdb11f078bf None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:44.303413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b20454-99a1-468b-b5c6-a58e0dc9c321 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:44.347419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274bb99-4c73-4d5e-9255-f3d6e920ad2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:44.348280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274bb99-4c73-4d5e-9255-f3d6e920ad2f None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:44.348556 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274bb99-4c73-4d5e-9255-f3d6e920ad2f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:44.349649 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274bb99-4c73-4d5e-9255-f3d6e920ad2f None admin] PATH_INFO: `/v3/users/d815fab9cbe84577b465bada61607286` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:44.364806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274bb99-4c73-4d5e-9255-f3d6e920ad2f None admin] RBAC: Authorizing `identity:get_user(user_id=d815fab9cbe84577b465bada61607286)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:44.365663 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c143e3-c1b3-48fd-83b1-bcdb11f078bf None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:44.366597 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274bb99-4c73-4d5e-9255-f3d6e920ad2f None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:44.368591 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 151/305] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:44 2022] GET /identity/v3/users/d815fab9cbe84577b465bada61607286 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:44.376642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274bb99-4c73-4d5e-9255-f3d6e920ad2f None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:44.417243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8ad9d-e15e-4b2b-ba88-b0f06c8a3669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:44.418324 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8ad9d-e15e-4b2b-ba88-b0f06c8a3669 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:44.418613 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8ad9d-e15e-4b2b-ba88-b0f06c8a3669 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:44.418893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8ad9d-e15e-4b2b-ba88-b0f06c8a3669 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:44.436833 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8ad9d-e15e-4b2b-ba88-b0f06c8a3669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-23e8ad9d-e15e-4b2b-ba88-b0f06c8a3669 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:44.452495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:44.453721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:44.454768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:44.454768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:44.454768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:44.454768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:44.454768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:44.454768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:44.454768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:44.454768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:44.454768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:44.454768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:44.454768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:44.454768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:44.454768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 155/306] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:12:44.462129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8ad9d-e15e-4b2b-ba88-b0f06c8a3669 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:44.515080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274bb99-4c73-4d5e-9255-f3d6e920ad2f None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:44.563994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf8a2e-cf01-42a6-85c1-21667a2dcc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:44.565082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf8a2e-cf01-42a6-85c1-21667a2dcc80 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:44.565463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf8a2e-cf01-42a6-85c1-21667a2dcc80 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:44.565840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf8a2e-cf01-42a6-85c1-21667a2dcc80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:44.567255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf8a2e-cf01-42a6-85c1-21667a2dcc80 None admin] RBAC: Adding query filter params (role.id=2a69f0e80033468f96c13ef5f64d094d, user.id=d815fab9cbe84577b465bada61607286, scope.system=all) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:44.567695 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf8a2e-cf01-42a6-85c1-21667a2dcc80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:44.569736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf8a2e-cf01-42a6-85c1-21667a2dcc80 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:44.576151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8ad9d-e15e-4b2b-ba88-b0f06c8a3669 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:44.580126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 152/307] 10.222.0.44 () {56 vars in 1308 bytes} [Mon May 9 04:12:44 2022] GET /identity/v3/role_assignments?user.id=d815fab9cbe84577b465bada61607286&scope.system=all&role.id=2a69f0e80033468f96c13ef5f64d094d => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:44.582054 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ce7a5-9149-4ee1-af62-dfcd3455fabd None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:44.582054 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ce7a5-9149-4ee1-af62-dfcd3455fabd None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:44.582054 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ce7a5-9149-4ee1-af62-dfcd3455fabd None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:44.583590 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 153/308] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:44.589478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e2c65-c846-45c9-a0c3-fcbbe6065964 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:44.590137 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e2c65-c846-45c9-a0c3-fcbbe6065964 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:44.590137 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e2c65-c846-45c9-a0c3-fcbbe6065964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:44.608427 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-859e2c65-c846-45c9-a0c3-fcbbe6065964 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:44.609556 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-859e2c65-c846-45c9-a0c3-fcbbe6065964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:44.620037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85d35a-84ac-49de-b02b-09fa278b7d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:44.621316 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85d35a-84ac-49de-b02b-09fa278b7d18 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:44.621747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85d35a-84ac-49de-b02b-09fa278b7d18 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:44.622169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85d35a-84ac-49de-b02b-09fa278b7d18 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:44.623079 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85d35a-84ac-49de-b02b-09fa278b7d18 None admin] RBAC: Adding query filter params (name=member) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:44.623602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85d35a-84ac-49de-b02b-09fa278b7d18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:44.625175 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85d35a-84ac-49de-b02b-09fa278b7d18 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:44.634473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 156/309] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:44.640779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85d35a-84ac-49de-b02b-09fa278b7d18 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:44.688070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 154/310] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:44.710343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e2c65-c846-45c9-a0c3-fcbbe6065964 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:44.751410 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85d35a-84ac-49de-b02b-09fa278b7d18 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:44.789775 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae24aa7-39f4-474d-ab4d-5b4c37f24973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:44.791014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae24aa7-39f4-474d-ab4d-5b4c37f24973 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:44.791534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae24aa7-39f4-474d-ab4d-5b4c37f24973 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:44.792124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae24aa7-39f4-474d-ab4d-5b4c37f24973 None admin] PATH_INFO: `/v3/users/cb3cff94a7c742d8a79e6c5c4244dad9` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:44.794859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae24aa7-39f4-474d-ab4d-5b4c37f24973 None admin] RBAC: Authorizing `identity:get_user(user_id=cb3cff94a7c742d8a79e6c5c4244dad9)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:44.797089 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae24aa7-39f4-474d-ab4d-5b4c37f24973 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:44.803675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 157/311] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:44 2022] GET /identity/v3/users/cb3cff94a7c742d8a79e6c5c4244dad9 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:44.804891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae24aa7-39f4-474d-ab4d-5b4c37f24973 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:44.833669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e2c65-c846-45c9-a0c3-fcbbe6065964 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:44.883331 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d286b2-a045-452f-b9b4-bd9464610a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:44.884395 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d286b2-a045-452f-b9b4-bd9464610a28 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:44.884645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d286b2-a045-452f-b9b4-bd9464610a28 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:44.884933 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d286b2-a045-452f-b9b4-bd9464610a28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:44.898340 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d286b2-a045-452f-b9b4-bd9464610a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-b0d286b2-a045-452f-b9b4-bd9464610a28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:44.907778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:44.908739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:44.909548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:44.909548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:44.909548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:44.909548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:44.909548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:44.909548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:44.909548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:44.909548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:44.909548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:44.909548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:44.909548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:44.909548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:44.910160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 155/312] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:12:44.915367 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae24aa7-39f4-474d-ab4d-5b4c37f24973 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:44.916255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d286b2-a045-452f-b9b4-bd9464610a28 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:44.950339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f40443-ce2c-4478-89e6-2d24858a94e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:44.951554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f40443-ce2c-4478-89e6-2d24858a94e4 None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:44.952137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f40443-ce2c-4478-89e6-2d24858a94e4 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:44.952629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f40443-ce2c-4478-89e6-2d24858a94e4 None admin] PATH_INFO: `/v3/system/users/cb3cff94a7c742d8a79e6c5c4244dad9/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:44.954697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f40443-ce2c-4478-89e6-2d24858a94e4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cb3cff94a7c742d8a79e6c5c4244dad9, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:44.956415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f40443-ce2c-4478-89e6-2d24858a94e4 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:44.969187 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 158/313] 10.222.0.44 () {58 vars in 1334 bytes} [Mon May 9 04:12:44 2022] PUT /identity/v3/system/users/cb3cff94a7c742d8a79e6c5c4244dad9/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:12:45.033488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d286b2-a045-452f-b9b4-bd9464610a28 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:45.071820 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3794a8e-d897-4c62-b684-08e992f28a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:45.072648 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3794a8e-d897-4c62-b684-08e992f28a66 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:45.072937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3794a8e-d897-4c62-b684-08e992f28a66 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:45.073215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3794a8e-d897-4c62-b684-08e992f28a66 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:45.073987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3794a8e-d897-4c62-b684-08e992f28a66 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:45.074295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3794a8e-d897-4c62-b684-08e992f28a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:45.075699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3794a8e-d897-4c62-b684-08e992f28a66 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:45.086467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 156/314] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:45.095963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f40443-ce2c-4478-89e6-2d24858a94e4 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:45.202461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f40443-ce2c-4478-89e6-2d24858a94e4 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:45.236123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98388318-4ab5-419c-863c-0ecdacab9842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:45.237001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98388318-4ab5-419c-863c-0ecdacab9842 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:45.237282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98388318-4ab5-419c-863c-0ecdacab9842 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:45.237553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98388318-4ab5-419c-863c-0ecdacab9842 None admin] PATH_INFO: `/v3/users/1ac46762402b426389ab68171593c437` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:45.238744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98388318-4ab5-419c-863c-0ecdacab9842 None admin] RBAC: Authorizing `identity:get_user(user_id=1ac46762402b426389ab68171593c437)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:45.240299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98388318-4ab5-419c-863c-0ecdacab9842 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:45.242212 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 159/315] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:45 2022] GET /identity/v3/users/1ac46762402b426389ab68171593c437 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:45.247423 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3794a8e-d897-4c62-b684-08e992f28a66 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:45.346413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3794a8e-d897-4c62-b684-08e992f28a66 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:45.378839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c54863-aec4-4c80-8aef-3d06f199a78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:45.379496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c54863-aec4-4c80-8aef-3d06f199a78c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:45.379776 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c54863-aec4-4c80-8aef-3d06f199a78c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:45.380077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c54863-aec4-4c80-8aef-3d06f199a78c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:45.448516 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c54863-aec4-4c80-8aef-3d06f199a78c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:45.449966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c54863-aec4-4c80-8aef-3d06f199a78c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:45.552007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 157/316] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:45.553461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98388318-4ab5-419c-863c-0ecdacab9842 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:45.652812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98388318-4ab5-419c-863c-0ecdacab9842 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:45.687058 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3dafc-94d7-4a0e-8663-c51a8fb13fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:45.688300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3dafc-94d7-4a0e-8663-c51a8fb13fab None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:45.688780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3dafc-94d7-4a0e-8663-c51a8fb13fab None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:45.689244 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3dafc-94d7-4a0e-8663-c51a8fb13fab None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:45.690594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3dafc-94d7-4a0e-8663-c51a8fb13fab None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:45.692249 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3dafc-94d7-4a0e-8663-c51a8fb13fab None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:45.694426 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 160/317] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:45 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:45.700301 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c54863-aec4-4c80-8aef-3d06f199a78c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:45.805029 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c54863-aec4-4c80-8aef-3d06f199a78c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:45.838878 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f19db-85b9-4862-8db7-d67c4d5a9e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:45.839538 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f19db-85b9-4862-8db7-d67c4d5a9e97 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:45.839876 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f19db-85b9-4862-8db7-d67c4d5a9e97 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:45.840180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f19db-85b9-4862-8db7-d67c4d5a9e97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:45.841072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f19db-85b9-4862-8db7-d67c4d5a9e97 None admin] RBAC: Adding query filter params (role.id=2a69f0e80033468f96c13ef5f64d094d, scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f, user.id=1ac46762402b426389ab68171593c437) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:45.841383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f19db-85b9-4862-8db7-d67c4d5a9e97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:45.843386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f19db-85b9-4862-8db7-d67c4d5a9e97 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:45.849765 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 158/318] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:12:45 2022] GET /identity/v3/role_assignments?user.id=1ac46762402b426389ab68171593c437&scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f&role.id=2a69f0e80033468f96c13ef5f64d094d => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:46.523199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2be2b-7ba9-4cfb-bd11-9e05282cdd2e None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:46.523592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2be2b-7ba9-4cfb-bd11-9e05282cdd2e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:46.523763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2be2b-7ba9-4cfb-bd11-9e05282cdd2e None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:46.525289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 161/319] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:46.530316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c293e24-0de6-4eac-a862-539fda4b2a9a None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:46.530604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c293e24-0de6-4eac-a862-539fda4b2a9a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:46.530877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c293e24-0de6-4eac-a862-539fda4b2a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:46.546960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-5c293e24-0de6-4eac-a862-539fda4b2a9a None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:46.548141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5c293e24-0de6-4eac-a862-539fda4b2a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:46.611870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 159/320] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:46.633807 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2be2b-7ba9-4cfb-bd11-9e05282cdd2e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:46.735604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2be2b-7ba9-4cfb-bd11-9e05282cdd2e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:46.769120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c714fcd-21bd-475b-8e77-440b4ba019f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:46.769921 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c714fcd-21bd-475b-8e77-440b4ba019f3 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:46.770198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c714fcd-21bd-475b-8e77-440b4ba019f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:46.770463 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c714fcd-21bd-475b-8e77-440b4ba019f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:46.771692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c714fcd-21bd-475b-8e77-440b4ba019f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:46.773146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c714fcd-21bd-475b-8e77-440b4ba019f3 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:46.774951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 162/321] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:12:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:46.780429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c293e24-0de6-4eac-a862-539fda4b2a9a None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:46.829500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4bb42-1267-47ff-ae4a-79e1163885b6 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:46.829785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4bb42-1267-47ff-ae4a-79e1163885b6 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:46.830066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4bb42-1267-47ff-ae4a-79e1163885b6 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:46.831420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 163/322] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:12:46.836219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152e3dc-763e-4b87-b27c-e776efbffb11 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:46.836506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152e3dc-763e-4b87-b27c-e776efbffb11 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:46.836782 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152e3dc-763e-4b87-b27c-e776efbffb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:46.852761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f152e3dc-763e-4b87-b27c-e776efbffb11 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:46.853938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f152e3dc-763e-4b87-b27c-e776efbffb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:46.877093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c293e24-0de6-4eac-a862-539fda4b2a9a None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:46.910478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085226e-863b-4af2-a5b9-f47999eee271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:46.911364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085226e-863b-4af2-a5b9-f47999eee271 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:46.911811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085226e-863b-4af2-a5b9-f47999eee271 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:46.912244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085226e-863b-4af2-a5b9-f47999eee271 None admin] PATH_INFO: `/v3/groups` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:46.913072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085226e-863b-4af2-a5b9-f47999eee271 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:46.914643 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085226e-863b-4af2-a5b9-f47999eee271 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:46.919605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 164/323] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:46.939972 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 160/324] 10.222.0.44 () {60 vars in 1134 bytes} [Mon May 9 04:12:46 2022] POST /identity/v3/groups => generated 226 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:12:46.943087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152e3dc-763e-4b87-b27c-e776efbffb11 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:47.044872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152e3dc-763e-4b87-b27c-e776efbffb11 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:47.079634 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591d426-c4ac-4906-9373-1a9bc194eea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:47.080478 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591d426-c4ac-4906-9373-1a9bc194eea7 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:47.080756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591d426-c4ac-4906-9373-1a9bc194eea7 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:47.081022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591d426-c4ac-4906-9373-1a9bc194eea7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:47.091607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591d426-c4ac-4906-9373-1a9bc194eea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-1591d426-c4ac-4906-9373-1a9bc194eea7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:47.099455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:47.100445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:47.101220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:47.101220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:47.101220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:47.101220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:47.101220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:47.101220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:47.101220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:47.101220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:47.101220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:47.101220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:47.101220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:47.101220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:47.101220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 165/325] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:47.110648 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085226e-863b-4af2-a5b9-f47999eee271 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:47.219882 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085226e-863b-4af2-a5b9-f47999eee271 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:47.253970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e00234-edbf-423c-8d1f-cb5ec5508ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:47.254650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e00234-edbf-423c-8d1f-cb5ec5508ed7 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:47.254932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e00234-edbf-423c-8d1f-cb5ec5508ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:47.255209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e00234-edbf-423c-8d1f-cb5ec5508ed7 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:47.255987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e00234-edbf-423c-8d1f-cb5ec5508ed7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:47.256289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e00234-edbf-423c-8d1f-cb5ec5508ed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:47.257713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e00234-edbf-423c-8d1f-cb5ec5508ed7 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:47.267201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 161/326] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:47.272630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591d426-c4ac-4906-9373-1a9bc194eea7 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:47.329634 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba588ce-863d-481d-8594-58eafcc26124 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:47.329948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba588ce-863d-481d-8594-58eafcc26124 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:47.330248 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba588ce-863d-481d-8594-58eafcc26124 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:47.331736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 162/327] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:47.337170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0793597-949c-442d-915d-b00f723674db None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:47.337720 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0793597-949c-442d-915d-b00f723674db None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:47.337720 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0793597-949c-442d-915d-b00f723674db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:47.357146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e0793597-949c-442d-915d-b00f723674db None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:47.359499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e0793597-949c-442d-915d-b00f723674db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:47.376692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591d426-c4ac-4906-9373-1a9bc194eea7 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:47.411125 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d293236-06c7-4a49-951f-e7b0a521e7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:47.412036 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d293236-06c7-4a49-951f-e7b0a521e7b7 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:47.412318 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d293236-06c7-4a49-951f-e7b0a521e7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:47.412590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d293236-06c7-4a49-951f-e7b0a521e7b7 None admin] PATH_INFO: `/v3/users/d815fab9cbe84577b465bada61607286` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:47.414510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d293236-06c7-4a49-951f-e7b0a521e7b7 None admin] RBAC: Authorizing `identity:get_user(user_id=d815fab9cbe84577b465bada61607286)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:47.415882 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d293236-06c7-4a49-951f-e7b0a521e7b7 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:47.417974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 166/328] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:47 2022] GET /identity/v3/users/d815fab9cbe84577b465bada61607286 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:47.423926 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d293236-06c7-4a49-951f-e7b0a521e7b7 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:47.443998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 163/329] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:47.462481 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0793597-949c-442d-915d-b00f723674db None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:47.525647 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d293236-06c7-4a49-951f-e7b0a521e7b7 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:47.558957 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7e441-42bd-43b6-864d-bd45903c803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:47.559817 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7e441-42bd-43b6-864d-bd45903c803b None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:47.560126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7e441-42bd-43b6-864d-bd45903c803b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:47.560402 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7e441-42bd-43b6-864d-bd45903c803b None admin] PATH_INFO: `/v3/system/users/d815fab9cbe84577b465bada61607286/roles/2a69f0e80033468f96c13ef5f64d094d` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:47.562829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7e441-42bd-43b6-864d-bd45903c803b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d815fab9cbe84577b465bada61607286, role_id=2a69f0e80033468f96c13ef5f64d094d)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:47.564284 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7e441-42bd-43b6-864d-bd45903c803b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:47.568569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0793597-949c-442d-915d-b00f723674db None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:47.573270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 167/330] 10.222.0.44 () {58 vars in 1334 bytes} [Mon May 9 04:12:47 2022] PUT /identity/v3/system/users/d815fab9cbe84577b465bada61607286/roles/2a69f0e80033468f96c13ef5f64d094d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:12:47.616280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27f8e0-355a-4ec8-8987-2805c55fbf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:47.617450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27f8e0-355a-4ec8-8987-2805c55fbf33 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:47.617968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27f8e0-355a-4ec8-8987-2805c55fbf33 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:47.618506 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27f8e0-355a-4ec8-8987-2805c55fbf33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:47.631767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27f8e0-355a-4ec8-8987-2805c55fbf33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-de27f8e0-355a-4ec8-8987-2805c55fbf33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:47.640610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:47.641558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:47.642433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:47.642433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:47.642433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:47.642433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:47.642433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:47.642433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:47.642433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:47.642433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:47.642433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:47.642433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:47.642433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:47.642433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:47.645141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 164/331] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:12:47.651393 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7e441-42bd-43b6-864d-bd45903c803b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:47.766787 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7e441-42bd-43b6-864d-bd45903c803b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:47.800628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656582e3-2738-493c-b61d-50581c9c2ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:47.801314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656582e3-2738-493c-b61d-50581c9c2ede None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:47.801603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656582e3-2738-493c-b61d-50581c9c2ede None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:47.801884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656582e3-2738-493c-b61d-50581c9c2ede None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:47.802641 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656582e3-2738-493c-b61d-50581c9c2ede None admin] RBAC: Adding query filter params (name=member) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:47.802950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656582e3-2738-493c-b61d-50581c9c2ede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:47.804367 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656582e3-2738-493c-b61d-50581c9c2ede None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:47.814662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 168/332] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:47.819318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27f8e0-355a-4ec8-8987-2805c55fbf33 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:47.934586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27f8e0-355a-4ec8-8987-2805c55fbf33 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:47.969692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb30c2-14ea-4fa2-b16c-f8a68b434d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:47.970577 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb30c2-14ea-4fa2-b16c-f8a68b434d6a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:47.970861 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb30c2-14ea-4fa2-b16c-f8a68b434d6a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:47.971138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb30c2-14ea-4fa2-b16c-f8a68b434d6a None admin] PATH_INFO: `/v3/users/cb3cff94a7c742d8a79e6c5c4244dad9` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:47.972393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb30c2-14ea-4fa2-b16c-f8a68b434d6a None admin] RBAC: Authorizing `identity:get_user(user_id=cb3cff94a7c742d8a79e6c5c4244dad9)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:47.973854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb30c2-14ea-4fa2-b16c-f8a68b434d6a None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:47.980314 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 165/333] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:47 2022] GET /identity/v3/users/cb3cff94a7c742d8a79e6c5c4244dad9 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:47.982398 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656582e3-2738-493c-b61d-50581c9c2ede None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:48.105470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656582e3-2738-493c-b61d-50581c9c2ede None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:48.144531 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9d86a-9be3-41cc-a724-f721451e06a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:48.145200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9d86a-9be3-41cc-a724-f721451e06a8 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:48.145489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9d86a-9be3-41cc-a724-f721451e06a8 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:48.145765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9d86a-9be3-41cc-a724-f721451e06a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:48.146671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9d86a-9be3-41cc-a724-f721451e06a8 None admin] RBAC: Adding query filter params (role.id=d8b94102152a46768d568f66794854f3, user.id=cb3cff94a7c742d8a79e6c5c4244dad9, scope.system=all) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:48.146983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9d86a-9be3-41cc-a724-f721451e06a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:48.148506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9d86a-9be3-41cc-a724-f721451e06a8 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:48.162430 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 169/334] 10.222.0.44 () {56 vars in 1308 bytes} [Mon May 9 04:12:47 2022] GET /identity/v3/role_assignments?user.id=cb3cff94a7c742d8a79e6c5c4244dad9&scope.system=all&role.id=d8b94102152a46768d568f66794854f3 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:50.050343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f0258-2223-44a0-aa9f-d4772c5ee585 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:50.050864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f0258-2223-44a0-aa9f-d4772c5ee585 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:50.050864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f0258-2223-44a0-aa9f-d4772c5ee585 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:50.052485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 166/335] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:50.057700 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1bb2b-e34f-48d8-a1fd-d9ff4b1d7f59 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:50.058566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1bb2b-e34f-48d8-a1fd-d9ff4b1d7f59 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:50.059049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1bb2b-e34f-48d8-a1fd-d9ff4b1d7f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:50.076392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e6859-04c4-44ad-bad3-d72b8616a9a1 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:50.077244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e6859-04c4-44ad-bad3-d72b8616a9a1 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:50.077492 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e6859-04c4-44ad-bad3-d72b8616a9a1 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:50.079382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 167/336] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:50.082627 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-7af1bb2b-e34f-48d8-a1fd-d9ff4b1d7f59 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:50.084025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-7af1bb2b-e34f-48d8-a1fd-d9ff4b1d7f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:50.084477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e86f92-ce1d-4c81-8815-897d0a97f6fc None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:50.084769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e86f92-ce1d-4c81-8815-897d0a97f6fc None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:50.085050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e86f92-ce1d-4c81-8815-897d0a97f6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:50.105920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-97e86f92-ce1d-4c81-8815-897d0a97f6fc None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:50.107100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-97e86f92-ce1d-4c81-8815-897d0a97f6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:50.167408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 170/337] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:50.189273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 168/338] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:50.206519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1bb2b-e34f-48d8-a1fd-d9ff4b1d7f59 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:50.228977 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e86f92-ce1d-4c81-8815-897d0a97f6fc None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:50.353501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e86f92-ce1d-4c81-8815-897d0a97f6fc None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:50.395433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a694ea-f5b3-42a8-adad-2994c1834e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:50.396404 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a694ea-f5b3-42a8-adad-2994c1834e57 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:50.396688 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a694ea-f5b3-42a8-adad-2994c1834e57 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:50.396963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a694ea-f5b3-42a8-adad-2994c1834e57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:50.418016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a694ea-f5b3-42a8-adad-2994c1834e57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-b4a694ea-f5b3-42a8-adad-2994c1834e57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:50.431917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:50.432891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:50.433704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:50.433704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:50.433704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:50.433704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:50.433704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:50.433704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:50.433704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:50.433704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:50.433704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:50.433704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:50.433704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:50.433704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:50.436548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 169/339] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:50.441845 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a694ea-f5b3-42a8-adad-2994c1834e57 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:50.442086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1bb2b-e34f-48d8-a1fd-d9ff4b1d7f59 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:50.528732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e9fc2-1a7e-4d9f-b69c-9e742fa14c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:50.530095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e9fc2-1a7e-4d9f-b69c-9e742fa14c51 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:50.530386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e9fc2-1a7e-4d9f-b69c-9e742fa14c51 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:50.530659 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e9fc2-1a7e-4d9f-b69c-9e742fa14c51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:50.553561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e9fc2-1a7e-4d9f-b69c-9e742fa14c51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-e62e9fc2-1a7e-4d9f-b69c-9e742fa14c51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:50.567701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:50.568667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:50.569435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:50.569435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:50.569435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:50.569435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:50.569435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:50.569435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:50.569435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:50.569435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:50.569435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:50.569435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:50.569435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:50.569435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:50.569435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 171/340] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:12:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:50.577171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e9fc2-1a7e-4d9f-b69c-9e742fa14c51 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:50.674023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a694ea-f5b3-42a8-adad-2994c1834e57 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:50.727789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e9fc2-1a7e-4d9f-b69c-9e742fa14c51 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:50.753510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283e4be-d913-4a71-b47b-92f7427681b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:50.754344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283e4be-d913-4a71-b47b-92f7427681b9 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:50.754741 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283e4be-d913-4a71-b47b-92f7427681b9 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:50.755500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283e4be-d913-4a71-b47b-92f7427681b9 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:50.756359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283e4be-d913-4a71-b47b-92f7427681b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:50.757267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283e4be-d913-4a71-b47b-92f7427681b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:50.759012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283e4be-d913-4a71-b47b-92f7427681b9 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:50.770458 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 170/341] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:50.771885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4a970-f6c7-438e-aad6-5d1f9c2e2e4f None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:50.772447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4a970-f6c7-438e-aad6-5d1f9c2e2e4f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:50.772731 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4a970-f6c7-438e-aad6-5d1f9c2e2e4f None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:50.774839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 171/342] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:50.776218 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4a970-f6c7-438e-aad6-5d1f9c2e2e4f None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:50.816593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcba60-fcf9-4672-a05f-4067410f8975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:50.817279 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcba60-fcf9-4672-a05f-4067410f8975 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:50.817563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcba60-fcf9-4672-a05f-4067410f8975 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:50.817838 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcba60-fcf9-4672-a05f-4067410f8975 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:50.818599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcba60-fcf9-4672-a05f-4067410f8975 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:50.818903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcba60-fcf9-4672-a05f-4067410f8975 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:50.820293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcba60-fcf9-4672-a05f-4067410f8975 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:50.830093 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 172/343] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:12:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:50.831380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52603ff6-0656-4cb8-9117-98b27f42b826 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:50.831667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52603ff6-0656-4cb8-9117-98b27f42b826 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:50.831961 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52603ff6-0656-4cb8-9117-98b27f42b826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:50.853700 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-52603ff6-0656-4cb8-9117-98b27f42b826 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:50.855088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-52603ff6-0656-4cb8-9117-98b27f42b826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:50.897141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4a970-f6c7-438e-aad6-5d1f9c2e2e4f None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:50.927293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 173/344] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:50.928118 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52603ff6-0656-4cb8-9117-98b27f42b826 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:50.953805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bafe6-79c6-4ed3-a00f-6ceb2a24f193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:50.954475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bafe6-79c6-4ed3-a00f-6ceb2a24f193 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:50.954763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bafe6-79c6-4ed3-a00f-6ceb2a24f193 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:50.955033 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bafe6-79c6-4ed3-a00f-6ceb2a24f193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:51.043955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52603ff6-0656-4cb8-9117-98b27f42b826 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:51.066021 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bafe6-79c6-4ed3-a00f-6ceb2a24f193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:51.067494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bafe6-79c6-4ed3-a00f-6ceb2a24f193 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:51.079200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af75bd-87fb-4319-ab1d-ec6138d59a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:51.079854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af75bd-87fb-4319-ab1d-ec6138d59a95 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:51.080159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af75bd-87fb-4319-ab1d-ec6138d59a95 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:51.080433 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af75bd-87fb-4319-ab1d-ec6138d59a95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:51.148038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af75bd-87fb-4319-ab1d-ec6138d59a95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:51.149503 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af75bd-87fb-4319-ab1d-ec6138d59a95 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:51.170720 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 172/345] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:51.171771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bafe6-79c6-4ed3-a00f-6ceb2a24f193 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:51.247701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 174/346] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:51.253041 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cf820-aa6f-48a9-b39b-47a3904e937d None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:51.253329 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cf820-aa6f-48a9-b39b-47a3904e937d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:51.253606 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cf820-aa6f-48a9-b39b-47a3904e937d None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:51.255127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 175/347] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:51.255843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cf820-aa6f-48a9-b39b-47a3904e937d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:51.269255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bafe6-79c6-4ed3-a00f-6ceb2a24f193 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:51.310799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e51d76-0de7-43f9-ac35-ac79c3bc59ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:51.312745 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e51d76-0de7-43f9-ac35-ac79c3bc59ba None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:51.313266 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e51d76-0de7-43f9-ac35-ac79c3bc59ba None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:51.313742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e51d76-0de7-43f9-ac35-ac79c3bc59ba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:51.332296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e51d76-0de7-43f9-ac35-ac79c3bc59ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-a9e51d76-0de7-43f9-ac35-ac79c3bc59ba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:51.341571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:51.342539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:51.343361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:51.343361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:51.343361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:51.343361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:51.343361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:51.343361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:51.343361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:51.343361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:51.343361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:51.343361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:51.343361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 04:12:51.343361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:51.345782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 173/348] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:51.346900 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e51d76-0de7-43f9-ac35-ac79c3bc59ba None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:51.369864 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cf820-aa6f-48a9-b39b-47a3904e937d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:51.419078 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3363c0-dd3a-4073-a752-e51fa64a5ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:51.420465 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3363c0-dd3a-4073-a752-e51fa64a5ef1 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:51.420797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3363c0-dd3a-4073-a752-e51fa64a5ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:51.421022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3363c0-dd3a-4073-a752-e51fa64a5ef1 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:51.422466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3363c0-dd3a-4073-a752-e51fa64a5ef1 None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:51.424076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3363c0-dd3a-4073-a752-e51fa64a5ef1 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:51.427460 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 176/349] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:51 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:51.427798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ef020-f4c2-414b-bcd5-0189dd76ac1d None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:51.428107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ef020-f4c2-414b-bcd5-0189dd76ac1d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:51.428381 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ef020-f4c2-414b-bcd5-0189dd76ac1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:51.447416 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-914ef020-f4c2-414b-bcd5-0189dd76ac1d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:51.449055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-914ef020-f4c2-414b-bcd5-0189dd76ac1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:51.462216 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e51d76-0de7-43f9-ac35-ac79c3bc59ba None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:51.509946 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b506a-4ee5-4770-9a41-cabcfc76cf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:51.510896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b506a-4ee5-4770-9a41-cabcfc76cf7c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:51.511316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b506a-4ee5-4770-9a41-cabcfc76cf7c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:51.511604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b506a-4ee5-4770-9a41-cabcfc76cf7c None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:51.513008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b506a-4ee5-4770-9a41-cabcfc76cf7c None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:51.515337 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b506a-4ee5-4770-9a41-cabcfc76cf7c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:51.518114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 174/350] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:51 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:51.518970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b506a-4ee5-4770-9a41-cabcfc76cf7c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:51.530342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 177/351] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:51.531176 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ef020-f4c2-414b-bcd5-0189dd76ac1d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:51.656685 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ef020-f4c2-414b-bcd5-0189dd76ac1d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:51.665395 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b506a-4ee5-4770-9a41-cabcfc76cf7c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:51.697584 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb6b7c-06e7-47f1-a289-95dc50401076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:51.698460 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb6b7c-06e7-47f1-a289-95dc50401076 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:51.698742 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb6b7c-06e7-47f1-a289-95dc50401076 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:51.699018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb6b7c-06e7-47f1-a289-95dc50401076 None admin] PATH_INFO: `/v3/groups/39e8fddd854f443f986b1055833619da` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:51.707791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f31b9-6350-4e6a-b0bd-de112b72b407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:51.708306 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb6b7c-06e7-47f1-a289-95dc50401076 None admin] RBAC: Authorizing `identity:get_group(group_id=39e8fddd854f443f986b1055833619da)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:51.708500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f31b9-6350-4e6a-b0bd-de112b72b407 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:51.708775 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f31b9-6350-4e6a-b0bd-de112b72b407 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:51.709052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f31b9-6350-4e6a-b0bd-de112b72b407 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:51.709773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb6b7c-06e7-47f1-a289-95dc50401076 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:51.709887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f31b9-6350-4e6a-b0bd-de112b72b407 None admin] RBAC: Adding query filter params (name=reader) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:51.710099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f31b9-6350-4e6a-b0bd-de112b72b407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:51.711528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 178/352] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:12:51 2022] GET /identity/v3/groups/39e8fddd854f443f986b1055833619da => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:51.711791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f31b9-6350-4e6a-b0bd-de112b72b407 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:51.712312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb6b7c-06e7-47f1-a289-95dc50401076 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:51.722412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 175/353] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:51.723119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f31b9-6350-4e6a-b0bd-de112b72b407 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:51.846142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f31b9-6350-4e6a-b0bd-de112b72b407 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:51.865410 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb6b7c-06e7-47f1-a289-95dc50401076 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:51.886195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b056244-df0f-4293-b868-b8e6446420fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:51.887214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b056244-df0f-4293-b868-b8e6446420fc None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:51.887330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b056244-df0f-4293-b868-b8e6446420fc None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:51.887544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b056244-df0f-4293-b868-b8e6446420fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:51.899504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b056244-df0f-4293-b868-b8e6446420fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-4b056244-df0f-4293-b868-b8e6446420fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:51.907204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:51.908126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:51.908932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:51.908932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:51.908932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:51.908932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:51.908932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:51.908932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:51.908932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:51.908932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:51.908932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:51.908932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:51.908932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:51.908932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:51.908932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 176/354] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:51.909602 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b056244-df0f-4293-b868-b8e6446420fc None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:51.913455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e03fea-0a55-4af3-abee-8cf7a4d5de47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:51.914334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e03fea-0a55-4af3-abee-8cf7a4d5de47 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:51.914619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e03fea-0a55-4af3-abee-8cf7a4d5de47 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:51.914893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e03fea-0a55-4af3-abee-8cf7a4d5de47 None admin] PATH_INFO: `/v3/groups/39e8fddd854f443f986b1055833619da` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:51.916140 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e03fea-0a55-4af3-abee-8cf7a4d5de47 None admin] RBAC: Authorizing `identity:get_group(group_id=39e8fddd854f443f986b1055833619da)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:51.917646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e03fea-0a55-4af3-abee-8cf7a4d5de47 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:51.919414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 179/355] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:12:51 2022] GET /identity/v3/groups/39e8fddd854f443f986b1055833619da => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:51.920080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e03fea-0a55-4af3-abee-8cf7a4d5de47 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:52.038049 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b056244-df0f-4293-b868-b8e6446420fc None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:52.044620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e03fea-0a55-4af3-abee-8cf7a4d5de47 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:52.071593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda98643-726f-4ded-9050-69887e84d322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:52.072282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda98643-726f-4ded-9050-69887e84d322 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:52.072567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda98643-726f-4ded-9050-69887e84d322 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:52.072834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda98643-726f-4ded-9050-69887e84d322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:52.073663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda98643-726f-4ded-9050-69887e84d322 None admin] RBAC: Adding query filter params (group.id=39e8fddd854f443f986b1055833619da, role.id=d8b94102152a46768d568f66794854f3, scope.project.id=37e07551c481458b8e025b20f23fd735) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:52.073971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda98643-726f-4ded-9050-69887e84d322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:52.075414 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda98643-726f-4ded-9050-69887e84d322 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:52.079782 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d5067-4156-4c48-ba8e-9c07210df3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:52.080684 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d5067-4156-4c48-ba8e-9c07210df3e8 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:52.080963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d5067-4156-4c48-ba8e-9c07210df3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:52.081239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d5067-4156-4c48-ba8e-9c07210df3e8 None admin] PATH_INFO: `/v3/users/d815fab9cbe84577b465bada61607286` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:52.082418 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d5067-4156-4c48-ba8e-9c07210df3e8 None admin] RBAC: Authorizing `identity:get_user(user_id=d815fab9cbe84577b465bada61607286)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:52.083118 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 177/356] 10.222.0.44 () {56 vars in 1376 bytes} [Mon May 9 04:12:51 2022] GET /identity/v3/role_assignments?group.id=39e8fddd854f443f986b1055833619da&scope.project.id=37e07551c481458b8e025b20f23fd735&role.id=d8b94102152a46768d568f66794854f3 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:52.083853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d5067-4156-4c48-ba8e-9c07210df3e8 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:52.084804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c97697-42d4-49ec-ac99-0862f68de1f2 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:52.085096 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c97697-42d4-49ec-ac99-0862f68de1f2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:52.085604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c97697-42d4-49ec-ac99-0862f68de1f2 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:52.085730 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 180/357] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:12:51 2022] GET /identity/v3/users/d815fab9cbe84577b465bada61607286 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:52.086254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d5067-4156-4c48-ba8e-9c07210df3e8 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:52.086756 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 178/358] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:52.087498 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c97697-42d4-49ec-ac99-0862f68de1f2 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:52.221480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d5067-4156-4c48-ba8e-9c07210df3e8 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:52.226371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c97697-42d4-49ec-ac99-0862f68de1f2 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:52.268175 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91569d32-97f3-456c-a776-7708d9537f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:52.268675 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d14217-ea58-49ff-8f28-3f9689768a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:52.268865 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91569d32-97f3-456c-a776-7708d9537f3d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:52.269151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91569d32-97f3-456c-a776-7708d9537f3d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:52.269429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91569d32-97f3-456c-a776-7708d9537f3d None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:52.269942 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d14217-ea58-49ff-8f28-3f9689768a34 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:52.270242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91569d32-97f3-456c-a776-7708d9537f3d None admin] RBAC: Adding query filter params (name=member) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:52.270528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d14217-ea58-49ff-8f28-3f9689768a34 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:52.270646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91569d32-97f3-456c-a776-7708d9537f3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:52.271043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d14217-ea58-49ff-8f28-3f9689768a34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:52.272050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d14217-ea58-49ff-8f28-3f9689768a34 None admin] RBAC: Adding query filter params (group.id=39e8fddd854f443f986b1055833619da, role.id=7600100dce614d9bb4d47b1e660b7bb2, scope.project.id=37e07551c481458b8e025b20f23fd735) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:52.272510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d14217-ea58-49ff-8f28-3f9689768a34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:52.272763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91569d32-97f3-456c-a776-7708d9537f3d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:52.274237 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d14217-ea58-49ff-8f28-3f9689768a34 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:52.282129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 179/359] 10.222.0.44 () {56 vars in 1376 bytes} [Mon May 9 04:12:52 2022] GET /identity/v3/role_assignments?group.id=39e8fddd854f443f986b1055833619da&scope.project.id=37e07551c481458b8e025b20f23fd735&role.id=7600100dce614d9bb4d47b1e660b7bb2 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:52.283636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16847056-c090-4e5d-8299-5fb28854e681 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:52.283852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 181/360] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:52.284261 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16847056-c090-4e5d-8299-5fb28854e681 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:52.284729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91569d32-97f3-456c-a776-7708d9537f3d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:52.284968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16847056-c090-4e5d-8299-5fb28854e681 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:52.292240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 180/361] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:52.293676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf65933-407e-4ba3-9b91-ba30989894d4 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:52.293968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf65933-407e-4ba3-9b91-ba30989894d4 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:52.294488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf65933-407e-4ba3-9b91-ba30989894d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:52.327667 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-fbf65933-407e-4ba3-9b91-ba30989894d4 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:52.329771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-fbf65933-407e-4ba3-9b91-ba30989894d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:52.413100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91569d32-97f3-456c-a776-7708d9537f3d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:52.430032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 181/362] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:52.431069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf65933-407e-4ba3-9b91-ba30989894d4 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:52.454821 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212f995-b72f-473c-a573-f00d6fc40df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:52.455930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212f995-b72f-473c-a573-f00d6fc40df7 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:52.456473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212f995-b72f-473c-a573-f00d6fc40df7 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:52.457084 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212f995-b72f-473c-a573-f00d6fc40df7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:52.458128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212f995-b72f-473c-a573-f00d6fc40df7 None admin] RBAC: Adding query filter params (role.id=2a69f0e80033468f96c13ef5f64d094d, user.id=d815fab9cbe84577b465bada61607286, scope.system=all) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:52.458703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212f995-b72f-473c-a573-f00d6fc40df7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:52.460456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212f995-b72f-473c-a573-f00d6fc40df7 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:52.474007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 182/363] 10.222.0.44 () {56 vars in 1308 bytes} [Mon May 9 04:12:52 2022] GET /identity/v3/role_assignments?user.id=d815fab9cbe84577b465bada61607286&scope.system=all&role.id=2a69f0e80033468f96c13ef5f64d094d => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:52.475754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1c9e9-2d9e-4f6e-96c3-a29b14ac3b5c None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:52.476280 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1c9e9-2d9e-4f6e-96c3-a29b14ac3b5c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:52.476782 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1c9e9-2d9e-4f6e-96c3-a29b14ac3b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:52.506370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-fcf1c9e9-2d9e-4f6e-96c3-a29b14ac3b5c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:52.508064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-fcf1c9e9-2d9e-4f6e-96c3-a29b14ac3b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:52.584051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf65933-407e-4ba3-9b91-ba30989894d4 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:52.596061 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 183/364] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:12:52.597390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1c9e9-2d9e-4f6e-96c3-a29b14ac3b5c None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:52.624463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f845151-737c-42ed-8656-ac9a7d405191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:52.625506 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f845151-737c-42ed-8656-ac9a7d405191 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:52.626011 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f845151-737c-42ed-8656-ac9a7d405191 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:52.626567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f845151-737c-42ed-8656-ac9a7d405191 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:52.702726 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f845151-737c-42ed-8656-ac9a7d405191 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:52.704887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f845151-737c-42ed-8656-ac9a7d405191 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:52.731959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1c9e9-2d9e-4f6e-96c3-a29b14ac3b5c None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:52.783955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db36be5-e6d1-4f22-917e-09b27cab89f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:52.784939 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db36be5-e6d1-4f22-917e-09b27cab89f1 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:52.785321 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db36be5-e6d1-4f22-917e-09b27cab89f1 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:52.785689 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db36be5-e6d1-4f22-917e-09b27cab89f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:52.800338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db36be5-e6d1-4f22-917e-09b27cab89f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-8db36be5-e6d1-4f22-917e-09b27cab89f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:52.809453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:52.810378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:52.811765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:52.811765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:52.811765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:52.811765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:52.811765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:52.811765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:52.811765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:52.811765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:52.811765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:52.811765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:52.811765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:52.811765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:52.811765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 184/365] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:12:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:52.814699 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db36be5-e6d1-4f22-917e-09b27cab89f1 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:52.839861 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 182/366] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:52.840988 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f845151-737c-42ed-8656-ac9a7d405191 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:52.942400 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db36be5-e6d1-4f22-917e-09b27cab89f1 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:52.970049 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f845151-737c-42ed-8656-ac9a7d405191 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:52.982192 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b933f28-996d-42d5-a7f1-f29c227859eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:52.983421 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b933f28-996d-42d5-a7f1-f29c227859eb None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:52.983937 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b933f28-996d-42d5-a7f1-f29c227859eb None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:52.984429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b933f28-996d-42d5-a7f1-f29c227859eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:52.996179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b933f28-996d-42d5-a7f1-f29c227859eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-3b933f28-996d-42d5-a7f1-f29c227859eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:53.005448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:53.006345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:53.007132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:53.007132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:53.007132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:53.007132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:53.007132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:53.007132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:53.007132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:53.007132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:53.007132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:53.007132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:53.007132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:53.007132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:53.009384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 185/367] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:12:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:53.010550 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b933f28-996d-42d5-a7f1-f29c227859eb None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:53.011920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd65b1-4c5d-4d54-a666-ef5821d3677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:53.012690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd65b1-4c5d-4d54-a666-ef5821d3677e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:53.013037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd65b1-4c5d-4d54-a666-ef5821d3677e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:53.013375 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd65b1-4c5d-4d54-a666-ef5821d3677e None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:53.014736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd65b1-4c5d-4d54-a666-ef5821d3677e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:53.015129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd65b1-4c5d-4d54-a666-ef5821d3677e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:53.017701 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd65b1-4c5d-4d54-a666-ef5821d3677e None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:53.031916 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 183/368] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:12:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:53.031916 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd65b1-4c5d-4d54-a666-ef5821d3677e None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:53.146335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b933f28-996d-42d5-a7f1-f29c227859eb None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:53.150818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd65b1-4c5d-4d54-a666-ef5821d3677e None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:53.182805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5aed4-afa7-4812-966a-1b4d7cf31d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:53.184345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5aed4-afa7-4812-966a-1b4d7cf31d3d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:53.184809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5aed4-afa7-4812-966a-1b4d7cf31d3d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:53.184913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324e680-dacb-4715-b8ac-308cf9e1afa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:53.185385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5aed4-afa7-4812-966a-1b4d7cf31d3d None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:53.185623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324e680-dacb-4715-b8ac-308cf9e1afa5 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:53.186016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324e680-dacb-4715-b8ac-308cf9e1afa5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:53.186368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324e680-dacb-4715-b8ac-308cf9e1afa5 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:53.187364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324e680-dacb-4715-b8ac-308cf9e1afa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:53.187618 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5aed4-afa7-4812-966a-1b4d7cf31d3d None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:53.187763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324e680-dacb-4715-b8ac-308cf9e1afa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:53.189301 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324e680-dacb-4715-b8ac-308cf9e1afa5 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:53.189627 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5aed4-afa7-4812-966a-1b4d7cf31d3d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:53.192362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 186/369] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:53 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:53.193370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5aed4-afa7-4812-966a-1b4d7cf31d3d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:53.199117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 184/370] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:53.200866 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324e680-dacb-4715-b8ac-308cf9e1afa5 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:53.302508 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324e680-dacb-4715-b8ac-308cf9e1afa5 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:53.317348 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5aed4-afa7-4812-966a-1b4d7cf31d3d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:53.340173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719cd93-4257-4be2-940e-0bd4ef3c1226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:53.341044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719cd93-4257-4be2-940e-0bd4ef3c1226 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:53.341331 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719cd93-4257-4be2-940e-0bd4ef3c1226 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:53.341590 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719cd93-4257-4be2-940e-0bd4ef3c1226 None admin] PATH_INFO: `/v3/groups/39e8fddd854f443f986b1055833619da` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:53.343129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719cd93-4257-4be2-940e-0bd4ef3c1226 None admin] RBAC: Authorizing `identity:get_group(group_id=39e8fddd854f443f986b1055833619da)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:53.344665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719cd93-4257-4be2-940e-0bd4ef3c1226 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:53.346407 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 185/371] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:12:53 2022] GET /identity/v3/groups/39e8fddd854f443f986b1055833619da => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:53.347164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719cd93-4257-4be2-940e-0bd4ef3c1226 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:53.355443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d002f-8e73-4aff-99a1-80087ecc51ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:53.356438 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d002f-8e73-4aff-99a1-80087ecc51ba None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:53.356923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d002f-8e73-4aff-99a1-80087ecc51ba None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:53.357397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d002f-8e73-4aff-99a1-80087ecc51ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:53.429282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d002f-8e73-4aff-99a1-80087ecc51ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:53.431077 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d002f-8e73-4aff-99a1-80087ecc51ba None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:53.454154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719cd93-4257-4be2-940e-0bd4ef3c1226 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:53.490909 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de716760-a984-464d-ab17-3ff8b11ecf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:53.491576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de716760-a984-464d-ab17-3ff8b11ecf99 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:53.491836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de716760-a984-464d-ab17-3ff8b11ecf99 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:53.492143 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de716760-a984-464d-ab17-3ff8b11ecf99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:53.536678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 187/372] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:53.537860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d002f-8e73-4aff-99a1-80087ecc51ba None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:53.561895 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de716760-a984-464d-ab17-3ff8b11ecf99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:53.563356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de716760-a984-464d-ab17-3ff8b11ecf99 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:53.661934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 186/373] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:12:53.662615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de716760-a984-464d-ab17-3ff8b11ecf99 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:53.675097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d002f-8e73-4aff-99a1-80087ecc51ba None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:53.715377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f1c99-b2aa-4fad-a087-1bdc2172f34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:53.716453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f1c99-b2aa-4fad-a087-1bdc2172f34a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:53.716971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f1c99-b2aa-4fad-a087-1bdc2172f34a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:53.717995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f1c99-b2aa-4fad-a087-1bdc2172f34a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:53.719039 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f1c99-b2aa-4fad-a087-1bdc2172f34a None admin] RBAC: Adding query filter params (group.id=39e8fddd854f443f986b1055833619da, role.id=d8b94102152a46768d568f66794854f3, scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:53.719563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f1c99-b2aa-4fad-a087-1bdc2172f34a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:53.721217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f1c99-b2aa-4fad-a087-1bdc2172f34a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:53.727061 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 188/374] 10.222.0.44 () {56 vars in 1376 bytes} [Mon May 9 04:12:53 2022] GET /identity/v3/role_assignments?group.id=39e8fddd854f443f986b1055833619da&scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f&role.id=d8b94102152a46768d568f66794854f3 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:53.728113 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f1c99-b2aa-4fad-a087-1bdc2172f34a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:53.762212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de716760-a984-464d-ab17-3ff8b11ecf99 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:53.800945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3da45-71b6-4e28-a7d6-6371321f1c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:53.801819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3da45-71b6-4e28-a7d6-6371321f1c13 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:53.802100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3da45-71b6-4e28-a7d6-6371321f1c13 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:53.802555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3da45-71b6-4e28-a7d6-6371321f1c13 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:53.803571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3da45-71b6-4e28-a7d6-6371321f1c13 None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:53.805075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3da45-71b6-4e28-a7d6-6371321f1c13 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:53.806856 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 187/375] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:53 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:53.812584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3da45-71b6-4e28-a7d6-6371321f1c13 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:53.844361 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f1c99-b2aa-4fad-a087-1bdc2172f34a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:53.883231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f5715-88e7-43e4-9ab1-cb982660d38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:53.884513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f5715-88e7-43e4-9ab1-cb982660d38b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:53.885001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f5715-88e7-43e4-9ab1-cb982660d38b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:53.885476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f5715-88e7-43e4-9ab1-cb982660d38b None admin] PATH_INFO: `/v3/projects/528420bd736843dc87d64954f4596bbd` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:53.887021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f5715-88e7-43e4-9ab1-cb982660d38b None admin] RBAC: Authorizing `identity:get_project(project_id=528420bd736843dc87d64954f4596bbd)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:53.888817 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f5715-88e7-43e4-9ab1-cb982660d38b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:53.891578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 189/376] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:53 2022] GET /identity/v3/projects/528420bd736843dc87d64954f4596bbd => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:53.897289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f5715-88e7-43e4-9ab1-cb982660d38b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:53.918464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3da45-71b6-4e28-a7d6-6371321f1c13 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:53.955300 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec4bdd-ccec-4bfc-8ed2-812e83472103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:53.956209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec4bdd-ccec-4bfc-8ed2-812e83472103 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:53.956488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec4bdd-ccec-4bfc-8ed2-812e83472103 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:53.956760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec4bdd-ccec-4bfc-8ed2-812e83472103 None admin] PATH_INFO: `/v3/groups/39e8fddd854f443f986b1055833619da` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:53.957995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec4bdd-ccec-4bfc-8ed2-812e83472103 None admin] RBAC: Authorizing `identity:get_group(group_id=39e8fddd854f443f986b1055833619da)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:53.959483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec4bdd-ccec-4bfc-8ed2-812e83472103 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:53.963933 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 188/377] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:12:53 2022] GET /identity/v3/groups/39e8fddd854f443f986b1055833619da => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:53.967939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec4bdd-ccec-4bfc-8ed2-812e83472103 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:54.062276 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f5715-88e7-43e4-9ab1-cb982660d38b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:54.070801 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec4bdd-ccec-4bfc-8ed2-812e83472103 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:54.102362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276cfde-ba8d-4960-b7f3-67a71fe82da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:54.103591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276cfde-ba8d-4960-b7f3-67a71fe82da5 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:54.104102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276cfde-ba8d-4960-b7f3-67a71fe82da5 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:54.104661 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276cfde-ba8d-4960-b7f3-67a71fe82da5 None admin] PATH_INFO: `/v3/groups/9134223cfe874829a524f380c32e8b44` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:54.108571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adf8f7-759c-455b-a563-be3cb17600f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:54.109228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adf8f7-759c-455b-a563-be3cb17600f1 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:54.109517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adf8f7-759c-455b-a563-be3cb17600f1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:54.109774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adf8f7-759c-455b-a563-be3cb17600f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:54.110725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adf8f7-759c-455b-a563-be3cb17600f1 None admin] RBAC: Adding query filter params (group.id=39e8fddd854f443f986b1055833619da, role.id=7600100dce614d9bb4d47b1e660b7bb2, scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:54.111031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adf8f7-759c-455b-a563-be3cb17600f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:54.111445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276cfde-ba8d-4960-b7f3-67a71fe82da5 None admin] RBAC: Authorizing `identity:get_group(group_id=9134223cfe874829a524f380c32e8b44)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:54.112517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adf8f7-759c-455b-a563-be3cb17600f1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:54.113359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276cfde-ba8d-4960-b7f3-67a71fe82da5 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:54.115535 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 190/378] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:12:53 2022] GET /identity/v3/groups/9134223cfe874829a524f380c32e8b44 => generated 229 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:54.119109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 189/379] 10.222.0.44 () {56 vars in 1376 bytes} [Mon May 9 04:12:53 2022] GET /identity/v3/role_assignments?group.id=39e8fddd854f443f986b1055833619da&scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f&role.id=7600100dce614d9bb4d47b1e660b7bb2 => generated 260 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:54.122791 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276cfde-ba8d-4960-b7f3-67a71fe82da5 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:54.235535 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276cfde-ba8d-4960-b7f3-67a71fe82da5 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:54.273335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c771f-6037-40ab-91e9-3963a1ba2161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:54.274007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c771f-6037-40ab-91e9-3963a1ba2161 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:54.274286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c771f-6037-40ab-91e9-3963a1ba2161 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:54.274556 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c771f-6037-40ab-91e9-3963a1ba2161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:54.275393 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c771f-6037-40ab-91e9-3963a1ba2161 None admin] RBAC: Adding query filter params (group.id=9134223cfe874829a524f380c32e8b44, role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=528420bd736843dc87d64954f4596bbd) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:54.275700 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c771f-6037-40ab-91e9-3963a1ba2161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:54.277185 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c771f-6037-40ab-91e9-3963a1ba2161 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:54.284171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 191/380] 10.222.0.44 () {56 vars in 1376 bytes} [Mon May 9 04:12:54 2022] GET /identity/v3/role_assignments?group.id=9134223cfe874829a524f380c32e8b44&scope.project.id=528420bd736843dc87d64954f4596bbd&role.id=e59113c0366049e889df3f55eed90c72 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:54.897691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d18f3f-1695-43e3-a348-3c18fd983055 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:54.898225 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d18f3f-1695-43e3-a348-3c18fd983055 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:54.898499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d18f3f-1695-43e3-a348-3c18fd983055 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:54.900242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 190/381] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:54.907034 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f20b47-6253-4fb6-b7c5-6cbd27747991 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:54.907323 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f20b47-6253-4fb6-b7c5-6cbd27747991 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:54.907605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f20b47-6253-4fb6-b7c5-6cbd27747991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:54.929099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-87f20b47-6253-4fb6-b7c5-6cbd27747991 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:54.930284 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-87f20b47-6253-4fb6-b7c5-6cbd27747991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:55.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 192/382] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:55.037217 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d18f3f-1695-43e3-a348-3c18fd983055 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:55.151558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d18f3f-1695-43e3-a348-3c18fd983055 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:55.190944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131003c-c4ef-465f-81e6-ee58158c9c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:55.191750 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131003c-c4ef-465f-81e6-ee58158c9c35 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:55.192052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131003c-c4ef-465f-81e6-ee58158c9c35 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:55.192324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131003c-c4ef-465f-81e6-ee58158c9c35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:55.209141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131003c-c4ef-465f-81e6-ee58158c9c35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-f131003c-c4ef-465f-81e6-ee58158c9c35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:55.216824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:55.218251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:55.219051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:55.219051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:55.219051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:55.219051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:55.219051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:55.219051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:55.219051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:55.219051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:55.219051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:55.219051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:55.219051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:55.219051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:55.219051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 191/383] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:55.224121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f20b47-6253-4fb6-b7c5-6cbd27747991 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:55.343615 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f20b47-6253-4fb6-b7c5-6cbd27747991 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:55.387595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92d646-2164-4e2a-838c-b5387ef8769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:55.388312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92d646-2164-4e2a-838c-b5387ef8769b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:55.388593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92d646-2164-4e2a-838c-b5387ef8769b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:55.389391 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92d646-2164-4e2a-838c-b5387ef8769b None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:55.390158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92d646-2164-4e2a-838c-b5387ef8769b None admin] RBAC: Adding query filter params (name=member) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:55.390654 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92d646-2164-4e2a-838c-b5387ef8769b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:55.392115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92d646-2164-4e2a-838c-b5387ef8769b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:55.401224 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 193/384] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:55.408717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131003c-c4ef-465f-81e6-ee58158c9c35 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:55.536811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7b79d-f4ce-4971-92dc-b931257802ea None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:55.537131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7b79d-f4ce-4971-92dc-b931257802ea None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:55.537362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7b79d-f4ce-4971-92dc-b931257802ea None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:55.538705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 194/385] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:12:55.543989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9127a3-2edf-46ff-8f46-a6508c8c11b4 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:55.544344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9127a3-2edf-46ff-8f46-a6508c8c11b4 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:55.544629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9127a3-2edf-46ff-8f46-a6508c8c11b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:55.576035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-5a9127a3-2edf-46ff-8f46-a6508c8c11b4 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:55.576035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5a9127a3-2edf-46ff-8f46-a6508c8c11b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:55.586401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131003c-c4ef-465f-81e6-ee58158c9c35 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:55.624428 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dcbe2-0793-42da-888d-c3cba85e80f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:55.625646 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dcbe2-0793-42da-888d-c3cba85e80f0 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:55.626167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dcbe2-0793-42da-888d-c3cba85e80f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:55.626658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dcbe2-0793-42da-888d-c3cba85e80f0 None admin] PATH_INFO: `/v3/groups/39e8fddd854f443f986b1055833619da` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:55.628081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19dcbe2-0793-42da-888d-c3cba85e80f0 None admin] RBAC: Authorizing `identity:get_group(group_id=39e8fddd854f443f986b1055833619da)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:55.636964 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19dcbe2-0793-42da-888d-c3cba85e80f0 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:55.639753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 192/386] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:12:55 2022] GET /identity/v3/groups/39e8fddd854f443f986b1055833619da => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:55.648817 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dcbe2-0793-42da-888d-c3cba85e80f0 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:55.747917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 195/387] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:55.765212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dcbe2-0793-42da-888d-c3cba85e80f0 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:55.805356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01746ab-5544-4cb2-b2ef-821329cbf932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:55.806626 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01746ab-5544-4cb2-b2ef-821329cbf932 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:55.807111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01746ab-5544-4cb2-b2ef-821329cbf932 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:55.807601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01746ab-5544-4cb2-b2ef-821329cbf932 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:55.809056 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01746ab-5544-4cb2-b2ef-821329cbf932 None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:55.810793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01746ab-5544-4cb2-b2ef-821329cbf932 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:55.813156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 193/388] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:55 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:55.818854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9127a3-2edf-46ff-8f46-a6508c8c11b4 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:55.820087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01746ab-5544-4cb2-b2ef-821329cbf932 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:55.936880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01746ab-5544-4cb2-b2ef-821329cbf932 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:55.984360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5a3f5-a6d7-4ebb-b1ef-2f3590b2684e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:55.985580 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5a3f5-a6d7-4ebb-b1ef-2f3590b2684e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:55.986085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5a3f5-a6d7-4ebb-b1ef-2f3590b2684e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:55.986575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5a3f5-a6d7-4ebb-b1ef-2f3590b2684e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:55.998937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5a3f5-a6d7-4ebb-b1ef-2f3590b2684e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-f0c5a3f5-a6d7-4ebb-b1ef-2f3590b2684e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:56.009055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:56.009973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:56.010777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:56.010777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:56.010777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:56.010777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:56.010777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:56.010777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:56.010777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:56.010777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:56.010777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:56.010777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:56.010777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:56.010777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:56.015165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 194/389] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:12:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:56.020325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5a3f5-a6d7-4ebb-b1ef-2f3590b2684e None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:56.073821 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9127a3-2edf-46ff-8f46-a6508c8c11b4 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:56.134727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5a3f5-a6d7-4ebb-b1ef-2f3590b2684e None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:56.157504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2decd3-ee83-4efc-9ade-aa66ecb2f9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:56.158637 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2decd3-ee83-4efc-9ade-aa66ecb2f9d2 None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:56.159133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2decd3-ee83-4efc-9ade-aa66ecb2f9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:56.159574 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2decd3-ee83-4efc-9ade-aa66ecb2f9d2 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735/groups/39e8fddd854f443f986b1055833619da/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:56.161823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2decd3-ee83-4efc-9ade-aa66ecb2f9d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e07551c481458b8e025b20f23fd735, group_id=39e8fddd854f443f986b1055833619da, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:56.163436 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2decd3-ee83-4efc-9ade-aa66ecb2f9d2 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:56.173926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6877a-e9cc-4a7c-b4d9-cfc54b6675bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:56.174615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6877a-e9cc-4a7c-b4d9-cfc54b6675bc None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:56.175056 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6877a-e9cc-4a7c-b4d9-cfc54b6675bc None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:56.175405 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6877a-e9cc-4a7c-b4d9-cfc54b6675bc None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:56.176181 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6877a-e9cc-4a7c-b4d9-cfc54b6675bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:56.176494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6877a-e9cc-4a7c-b4d9-cfc54b6675bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:56.178221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6877a-e9cc-4a7c-b4d9-cfc54b6675bc None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:56.186004 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 196/390] 10.222.0.44 () {58 vars in 1442 bytes} [Mon May 9 04:12:55 2022] PUT /identity/v3/projects/37e07551c481458b8e025b20f23fd735/groups/39e8fddd854f443f986b1055833619da/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:12:56.188178 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 195/391] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:12:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:56.193628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2decd3-ee83-4efc-9ade-aa66ecb2f9d2 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:56.309945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2decd3-ee83-4efc-9ade-aa66ecb2f9d2 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:56.353629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975abc2-0b0e-4a42-be4e-e27eb56725ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:56.355271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975abc2-0b0e-4a42-be4e-e27eb56725ee None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:56.355763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975abc2-0b0e-4a42-be4e-e27eb56725ee None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:56.356258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975abc2-0b0e-4a42-be4e-e27eb56725ee None admin] PATH_INFO: `/v3/groups/39e8fddd854f443f986b1055833619da` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:56.360932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975abc2-0b0e-4a42-be4e-e27eb56725ee None admin] RBAC: Authorizing `identity:get_group(group_id=39e8fddd854f443f986b1055833619da)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:56.362686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975abc2-0b0e-4a42-be4e-e27eb56725ee None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:56.365009 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 197/392] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:12:56 2022] GET /identity/v3/groups/39e8fddd854f443f986b1055833619da => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:56.370127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6877a-e9cc-4a7c-b4d9-cfc54b6675bc None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:56.497913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6877a-e9cc-4a7c-b4d9-cfc54b6675bc None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:56.536232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d20d3c-381d-47ff-aa33-d5bc72ba4944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:56.537112 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d20d3c-381d-47ff-aa33-d5bc72ba4944 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:56.537399 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d20d3c-381d-47ff-aa33-d5bc72ba4944 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:56.537676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d20d3c-381d-47ff-aa33-d5bc72ba4944 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:56.538887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d20d3c-381d-47ff-aa33-d5bc72ba4944 None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:56.540363 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d20d3c-381d-47ff-aa33-d5bc72ba4944 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:56.542412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 196/393] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:56 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:56.548471 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975abc2-0b0e-4a42-be4e-e27eb56725ee None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:56.652438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e98da-6ff7-448f-b4a5-e8bb7e738cd7 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:56.652754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e98da-6ff7-448f-b4a5-e8bb7e738cd7 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:56.652997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e98da-6ff7-448f-b4a5-e8bb7e738cd7 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:56.654844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 197/394] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:56.655024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975abc2-0b0e-4a42-be4e-e27eb56725ee None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:56.659981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a490b-1a14-45a4-9e35-31f4847ba6f1 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:56.660273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a490b-1a14-45a4-9e35-31f4847ba6f1 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:56.660552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a490b-1a14-45a4-9e35-31f4847ba6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:56.681182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-b84a490b-1a14-45a4-9e35-31f4847ba6f1 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:56.682343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b84a490b-1a14-45a4-9e35-31f4847ba6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:56.701078 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51704de-81bf-4897-ae75-6d13f33c8144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:56.702014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51704de-81bf-4897-ae75-6d13f33c8144 None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:56.702292 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51704de-81bf-4897-ae75-6d13f33c8144 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:56.702562 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51704de-81bf-4897-ae75-6d13f33c8144 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735/groups/39e8fddd854f443f986b1055833619da/roles/7600100dce614d9bb4d47b1e660b7bb2` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:56.707022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51704de-81bf-4897-ae75-6d13f33c8144 None admin] RBAC: Authorizing `identity:create_grant(project_id=37e07551c481458b8e025b20f23fd735, group_id=39e8fddd854f443f986b1055833619da, role_id=7600100dce614d9bb4d47b1e660b7bb2)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:56.708504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51704de-81bf-4897-ae75-6d13f33c8144 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:56.738460 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 198/395] 10.222.0.44 () {58 vars in 1442 bytes} [Mon May 9 04:12:56 2022] PUT /identity/v3/projects/37e07551c481458b8e025b20f23fd735/groups/39e8fddd854f443f986b1055833619da/roles/7600100dce614d9bb4d47b1e660b7bb2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:12:56.775147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 198/396] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:56.799877 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51704de-81bf-4897-ae75-6d13f33c8144 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:56.937469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51704de-81bf-4897-ae75-6d13f33c8144 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:56.988711 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104282c-a2c1-4fe6-82d5-aaba80784476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:56.989545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104282c-a2c1-4fe6-82d5-aaba80784476 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:56.989827 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104282c-a2c1-4fe6-82d5-aaba80784476 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:56.990097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104282c-a2c1-4fe6-82d5-aaba80784476 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:57.002035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104282c-a2c1-4fe6-82d5-aaba80784476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-b104282c-a2c1-4fe6-82d5-aaba80784476 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:57.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:57.010932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:57.012198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:57.012198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:57.012198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:57.012198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:57.012198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:57.012198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:57.012198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:57.012198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:57.012198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:57.012198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:57.012198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:57.012198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:57.012198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 199/397] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:57.019933 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a490b-1a14-45a4-9e35-31f4847ba6f1 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:57.146806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a490b-1a14-45a4-9e35-31f4847ba6f1 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:57.188561 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3624ea-06b2-4d86-ac75-106fc0568998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:57.189254 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3624ea-06b2-4d86-ac75-106fc0568998 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:57.189538 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3624ea-06b2-4d86-ac75-106fc0568998 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:57.189815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3624ea-06b2-4d86-ac75-106fc0568998 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:57.190602 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3624ea-06b2-4d86-ac75-106fc0568998 None admin] RBAC: Adding query filter params (name=member) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:57.190912 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3624ea-06b2-4d86-ac75-106fc0568998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:57.192370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3624ea-06b2-4d86-ac75-106fc0568998 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:57.205348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 199/398] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:57.211234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104282c-a2c1-4fe6-82d5-aaba80784476 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:57.315139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104282c-a2c1-4fe6-82d5-aaba80784476 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:57.348407 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea71f5-b4a5-4a84-8e75-ca52d17fdd6a None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:57.348757 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea71f5-b4a5-4a84-8e75-ca52d17fdd6a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:57.348973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea71f5-b4a5-4a84-8e75-ca52d17fdd6a None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:57.350576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 200/399] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:12:57.351871 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635735c-035a-4943-8241-bd1cdd2029c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:57.353001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635735c-035a-4943-8241-bd1cdd2029c1 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:57.353500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635735c-035a-4943-8241-bd1cdd2029c1 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:57.353974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635735c-035a-4943-8241-bd1cdd2029c1 None admin] PATH_INFO: `/v3/groups/39e8fddd854f443f986b1055833619da` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:57.356157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fd582-4d38-42cb-9a18-c43fb57b1762 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:57.356157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fd582-4d38-42cb-9a18-c43fb57b1762 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:57.356157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fd582-4d38-42cb-9a18-c43fb57b1762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:57.357164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635735c-035a-4943-8241-bd1cdd2029c1 None admin] RBAC: Authorizing `identity:get_group(group_id=39e8fddd854f443f986b1055833619da)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:57.359608 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635735c-035a-4943-8241-bd1cdd2029c1 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:57.361628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 200/400] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:12:57 2022] GET /identity/v3/groups/39e8fddd854f443f986b1055833619da => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:57.367647 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635735c-035a-4943-8241-bd1cdd2029c1 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:57.396948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e15fd582-4d38-42cb-9a18-c43fb57b1762 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:57.398150 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e15fd582-4d38-42cb-9a18-c43fb57b1762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:57.481538 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 201/401] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:57.483384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb1707-caed-4acf-9386-c56defd63b83 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:57.483692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb1707-caed-4acf-9386-c56defd63b83 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:57.488041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb1707-caed-4acf-9386-c56defd63b83 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:57.489368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 202/402] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:57.490820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635735c-035a-4943-8241-bd1cdd2029c1 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:57.494570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773772d-627c-44ee-9284-42e5f154b602 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:57.494857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773772d-627c-44ee-9284-42e5f154b602 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:57.495135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773772d-627c-44ee-9284-42e5f154b602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:57.526855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-8773772d-627c-44ee-9284-42e5f154b602 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:57.528502 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8773772d-627c-44ee-9284-42e5f154b602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:57.535000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905333c0-ba1b-4a6d-9310-0e4e036ff26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:57.536388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905333c0-ba1b-4a6d-9310-0e4e036ff26f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:57.537297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905333c0-ba1b-4a6d-9310-0e4e036ff26f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:57.537803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905333c0-ba1b-4a6d-9310-0e4e036ff26f None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:57.540645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905333c0-ba1b-4a6d-9310-0e4e036ff26f None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:57.542699 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905333c0-ba1b-4a6d-9310-0e4e036ff26f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:57.546054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 201/403] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:57 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:57.547390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905333c0-ba1b-4a6d-9310-0e4e036ff26f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:57.597206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 203/404] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:57.598070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773772d-627c-44ee-9284-42e5f154b602 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:57.680077 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905333c0-ba1b-4a6d-9310-0e4e036ff26f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:57.714001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773772d-627c-44ee-9284-42e5f154b602 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:57.719053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499a3d9-2612-49b9-a1ab-34ccc44c3988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:57.720243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499a3d9-2612-49b9-a1ab-34ccc44c3988 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:57.720736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499a3d9-2612-49b9-a1ab-34ccc44c3988 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:57.721191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499a3d9-2612-49b9-a1ab-34ccc44c3988 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:57.740404 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499a3d9-2612-49b9-a1ab-34ccc44c3988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-3499a3d9-2612-49b9-a1ab-34ccc44c3988 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:57.754275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:57.755174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:57.756298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:57.756298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:57.756298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:57.756298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:57.756298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:57.756298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:57.756298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:57.756298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:57.756298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:57.756298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:57.756298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:12:57.756298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:57.759201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f578d23-6d9b-4ef0-8bd9-21eb290cbc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:57.760502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 202/405] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:12:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:57.761364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f578d23-6d9b-4ef0-8bd9-21eb290cbc84 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:57.761484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499a3d9-2612-49b9-a1ab-34ccc44c3988 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:57.762025 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f578d23-6d9b-4ef0-8bd9-21eb290cbc84 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:57.762609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f578d23-6d9b-4ef0-8bd9-21eb290cbc84 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f/groups/39e8fddd854f443f986b1055833619da/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:57.765903 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f578d23-6d9b-4ef0-8bd9-21eb290cbc84 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f, group_id=39e8fddd854f443f986b1055833619da, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:57.768692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f578d23-6d9b-4ef0-8bd9-21eb290cbc84 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:57.786767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 204/406] 10.222.0.44 () {58 vars in 1442 bytes} [Mon May 9 04:12:57 2022] PUT /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f/groups/39e8fddd854f443f986b1055833619da/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:12:57.788260 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f578d23-6d9b-4ef0-8bd9-21eb290cbc84 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:57.917178 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f578d23-6d9b-4ef0-8bd9-21eb290cbc84 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:57.959092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2c13b-55ed-42e4-991e-0c6d6dae1e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:57.960167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2c13b-55ed-42e4-991e-0c6d6dae1e41 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:57.960628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2c13b-55ed-42e4-991e-0c6d6dae1e41 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:57.961102 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2c13b-55ed-42e4-991e-0c6d6dae1e41 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:57.962044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2c13b-55ed-42e4-991e-0c6d6dae1e41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:57.962527 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2c13b-55ed-42e4-991e-0c6d6dae1e41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:57.965184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2c13b-55ed-42e4-991e-0c6d6dae1e41 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:57.976146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 205/407] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:57.984185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2c13b-55ed-42e4-991e-0c6d6dae1e41 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:57.993371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499a3d9-2612-49b9-a1ab-34ccc44c3988 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:58.031018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164344ea-0bc0-41a5-b907-28f19965a45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:58.032237 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164344ea-0bc0-41a5-b907-28f19965a45e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:58.032237 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164344ea-0bc0-41a5-b907-28f19965a45e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:58.032432 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164344ea-0bc0-41a5-b907-28f19965a45e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:58.044282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164344ea-0bc0-41a5-b907-28f19965a45e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-164344ea-0bc0-41a5-b907-28f19965a45e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:58.053652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:58.054575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:58.055600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:58.055600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:58.055600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:58.055600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:58.055600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:58.055600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:58.055600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:58.055600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:58.055600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:58.055600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:58.055600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:12:58.055600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:12:58.055600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 203/408] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:12:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:58.068931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164344ea-0bc0-41a5-b907-28f19965a45e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:58.104484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2c13b-55ed-42e4-991e-0c6d6dae1e41 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:58.149144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3b750-9c6c-4dcd-a2c8-5be0cb6d425a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:58.149999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3b750-9c6c-4dcd-a2c8-5be0cb6d425a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:58.150275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3b750-9c6c-4dcd-a2c8-5be0cb6d425a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:58.150544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3b750-9c6c-4dcd-a2c8-5be0cb6d425a None admin] PATH_INFO: `/v3/groups/9134223cfe874829a524f380c32e8b44` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:58.156617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3b750-9c6c-4dcd-a2c8-5be0cb6d425a None admin] RBAC: Authorizing `identity:get_group(group_id=9134223cfe874829a524f380c32e8b44)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:58.158064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3b750-9c6c-4dcd-a2c8-5be0cb6d425a None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:58.160019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 206/409] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:12:57 2022] GET /identity/v3/groups/9134223cfe874829a524f380c32e8b44 => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:58.164943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3b750-9c6c-4dcd-a2c8-5be0cb6d425a None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:58.207877 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164344ea-0bc0-41a5-b907-28f19965a45e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:58.244821 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a2df4-8030-4f70-8ab2-e45f554eff16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:58.245840 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a2df4-8030-4f70-8ab2-e45f554eff16 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:58.247099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a2df4-8030-4f70-8ab2-e45f554eff16 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:58.247765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a2df4-8030-4f70-8ab2-e45f554eff16 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:58.250016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a2df4-8030-4f70-8ab2-e45f554eff16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:58.250539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a2df4-8030-4f70-8ab2-e45f554eff16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:58.252399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a2df4-8030-4f70-8ab2-e45f554eff16 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:58.270543 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 204/410] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:12:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:58.277493 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a2df4-8030-4f70-8ab2-e45f554eff16 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:58.324560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3b750-9c6c-4dcd-a2c8-5be0cb6d425a None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:58.384642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b83ad-1e70-474b-bf1b-2a9553937d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:58.385550 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b83ad-1e70-474b-bf1b-2a9553937d84 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:58.385832 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b83ad-1e70-474b-bf1b-2a9553937d84 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:58.386103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b83ad-1e70-474b-bf1b-2a9553937d84 None admin] PATH_INFO: `/v3/projects/528420bd736843dc87d64954f4596bbd` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:58.387335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b83ad-1e70-474b-bf1b-2a9553937d84 None admin] RBAC: Authorizing `identity:get_project(project_id=528420bd736843dc87d64954f4596bbd)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:58.389316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b83ad-1e70-474b-bf1b-2a9553937d84 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:58.400355 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 207/411] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:58 2022] GET /identity/v3/projects/528420bd736843dc87d64954f4596bbd => generated 344 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:58.406429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b83ad-1e70-474b-bf1b-2a9553937d84 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:58.423284 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a2df4-8030-4f70-8ab2-e45f554eff16 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:58.459149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ff6dc-c75b-46e9-b9ec-ec2b7cd22c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:58.460013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ff6dc-c75b-46e9-b9ec-ec2b7cd22c7a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:58.460296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ff6dc-c75b-46e9-b9ec-ec2b7cd22c7a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:58.460568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ff6dc-c75b-46e9-b9ec-ec2b7cd22c7a None admin] PATH_INFO: `/v3/groups/39e8fddd854f443f986b1055833619da` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:58.461765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ff6dc-c75b-46e9-b9ec-ec2b7cd22c7a None admin] RBAC: Authorizing `identity:get_group(group_id=39e8fddd854f443f986b1055833619da)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:58.463247 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ff6dc-c75b-46e9-b9ec-ec2b7cd22c7a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:58.465303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 205/412] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:12:58 2022] GET /identity/v3/groups/39e8fddd854f443f986b1055833619da => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:58.471631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ff6dc-c75b-46e9-b9ec-ec2b7cd22c7a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:58.571846 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ff6dc-c75b-46e9-b9ec-ec2b7cd22c7a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:58.605690 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d6d8c-87a7-4679-ba98-70db81633aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:58.606555 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d6d8c-87a7-4679-ba98-70db81633aae None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:58.606831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d6d8c-87a7-4679-ba98-70db81633aae None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:58.607110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d6d8c-87a7-4679-ba98-70db81633aae None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:58.608965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d6d8c-87a7-4679-ba98-70db81633aae None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:58.610391 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d6d8c-87a7-4679-ba98-70db81633aae None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:58.612404 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 206/413] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:12:58 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:12:58.617768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d6d8c-87a7-4679-ba98-70db81633aae None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:58.638198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b83ad-1e70-474b-bf1b-2a9553937d84 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:58.672600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e3abe-2c8f-41c0-b5eb-6de14de51a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:58.673496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e3abe-2c8f-41c0-b5eb-6de14de51a74 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:58.673778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e3abe-2c8f-41c0-b5eb-6de14de51a74 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:58.674052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e3abe-2c8f-41c0-b5eb-6de14de51a74 None admin] PATH_INFO: `/v3/projects/528420bd736843dc87d64954f4596bbd/groups/9134223cfe874829a524f380c32e8b44/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:58.676041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e3abe-2c8f-41c0-b5eb-6de14de51a74 None admin] RBAC: Authorizing `identity:create_grant(project_id=528420bd736843dc87d64954f4596bbd, group_id=9134223cfe874829a524f380c32e8b44, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:58.677550 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e3abe-2c8f-41c0-b5eb-6de14de51a74 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:58.710674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 208/414] 10.222.0.44 () {58 vars in 1442 bytes} [Mon May 9 04:12:58 2022] PUT /identity/v3/projects/528420bd736843dc87d64954f4596bbd/groups/9134223cfe874829a524f380c32e8b44/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:12:58.720030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d6d8c-87a7-4679-ba98-70db81633aae None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:58.758612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc563e7d-9dc9-4b3e-91e9-7069e24459db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:58.759667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc563e7d-9dc9-4b3e-91e9-7069e24459db None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:58.760162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc563e7d-9dc9-4b3e-91e9-7069e24459db None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:58.760502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc563e7d-9dc9-4b3e-91e9-7069e24459db None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f/groups/39e8fddd854f443f986b1055833619da/roles/7600100dce614d9bb4d47b1e660b7bb2` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:58.763166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc563e7d-9dc9-4b3e-91e9-7069e24459db None admin] RBAC: Authorizing `identity:create_grant(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f, group_id=39e8fddd854f443f986b1055833619da, role_id=7600100dce614d9bb4d47b1e660b7bb2)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:58.765155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc563e7d-9dc9-4b3e-91e9-7069e24459db None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:58.781406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 207/415] 10.222.0.44 () {58 vars in 1442 bytes} [Mon May 9 04:12:58 2022] PUT /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f/groups/39e8fddd854f443f986b1055833619da/roles/7600100dce614d9bb4d47b1e660b7bb2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:12:59.173176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc8113-abce-4c34-aa53-81f763691e64 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:59.174195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc8113-abce-4c34-aa53-81f763691e64 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:59.174686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc8113-abce-4c34-aa53-81f763691e64 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:59.176535 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 209/416] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:59.182457 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1180050-5beb-4c9a-b0f3-bd8f00b9f735 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:59.183025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1180050-5beb-4c9a-b0f3-bd8f00b9f735 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:59.183522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1180050-5beb-4c9a-b0f3-bd8f00b9f735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:59.212919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f1180050-5beb-4c9a-b0f3-bd8f00b9f735 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:59.214469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f1180050-5beb-4c9a-b0f3-bd8f00b9f735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:59.301778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 208/417] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:12:59.342481 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc8113-abce-4c34-aa53-81f763691e64 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:59.468286 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc8113-abce-4c34-aa53-81f763691e64 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:59.503026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b10fa-faaf-4ff1-b7ab-ab7adb32f371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:59.504230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b10fa-faaf-4ff1-b7ab-ab7adb32f371 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:59.504718 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b10fa-faaf-4ff1-b7ab-ab7adb32f371 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:59.505217 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b10fa-faaf-4ff1-b7ab-ab7adb32f371 None admin] PATH_INFO: `/v3/roles/member` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:59.522700 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b10fa-faaf-4ff1-b7ab-ab7adb32f371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-926b10fa-faaf-4ff1-b7ab-ab7adb32f371 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:12:59.531031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:12:59.531939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:12:59.532734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:12:59.532734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:12:59.532734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:12:59.532734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:12:59.532734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:12:59.532734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:12:59.532734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:12:59.532734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:12:59.532734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:12:59.532734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:12:59.532734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:12:59.532734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:12:59.535171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 210/418] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:12:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:12:59.541430 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1180050-5beb-4c9a-b0f3-bd8f00b9f735 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:59.666150 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1180050-5beb-4c9a-b0f3-bd8f00b9f735 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:59.701304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9a2dd-8aa0-4e68-9b63-e5f623545d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:59.701996 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9a2dd-8aa0-4e68-9b63-e5f623545d3b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:59.702277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9a2dd-8aa0-4e68-9b63-e5f623545d3b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:59.702553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9a2dd-8aa0-4e68-9b63-e5f623545d3b None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:59.703548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9a2dd-8aa0-4e68-9b63-e5f623545d3b None admin] RBAC: Adding query filter params (name=member) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:12:59.703858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9a2dd-8aa0-4e68-9b63-e5f623545d3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:12:59.705253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9a2dd-8aa0-4e68-9b63-e5f623545d3b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:12:59.713895 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 209/419] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:12:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:12:59.719669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b10fa-faaf-4ff1-b7ab-ab7adb32f371 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:12:59.841112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818642c-74f8-4338-9470-121a47c8776e None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:59.841455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818642c-74f8-4338-9470-121a47c8776e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:59.841679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818642c-74f8-4338-9470-121a47c8776e None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:59.843522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 210/420] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:12:59.848860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cade89-3cf6-4c49-bc38-b8de0f928d54 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:59.849156 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cade89-3cf6-4c49-bc38-b8de0f928d54 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:59.849435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cade89-3cf6-4c49-bc38-b8de0f928d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:12:59.878726 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b10fa-faaf-4ff1-b7ab-ab7adb32f371 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:12:59.884907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f1cade89-3cf6-4c49-bc38-b8de0f928d54 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:12:59.886104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f1cade89-3cf6-4c49-bc38-b8de0f928d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:12:59.914377 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ced00-4af2-4671-ba25-23aaa0a8033e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:12:59.915039 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ced00-4af2-4671-ba25-23aaa0a8033e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:12:59.915324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ced00-4af2-4671-ba25-23aaa0a8033e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:12:59.915604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ced00-4af2-4671-ba25-23aaa0a8033e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:00.025422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ced00-4af2-4671-ba25-23aaa0a8033e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:00.027243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ced00-4af2-4671-ba25-23aaa0a8033e None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:00.046121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 211/421] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:00.081615 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cade89-3cf6-4c49-bc38-b8de0f928d54 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:00.138696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 211/422] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:12:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:13:00.140749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ced00-4af2-4671-ba25-23aaa0a8033e None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:00.250252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ced00-4af2-4671-ba25-23aaa0a8033e None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:00.287249 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cfb63-30d4-4bf3-9207-7667c807f34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:00.288070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cfb63-30d4-4bf3-9207-7667c807f34f None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:00.288358 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cfb63-30d4-4bf3-9207-7667c807f34f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:00.288636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cfb63-30d4-4bf3-9207-7667c807f34f None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:00.289831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cfb63-30d4-4bf3-9207-7667c807f34f None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:00.291398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cfb63-30d4-4bf3-9207-7667c807f34f None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:00.297752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 212/423] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:13:00 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:00.299012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cfb63-30d4-4bf3-9207-7667c807f34f None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:00.323920 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cade89-3cf6-4c49-bc38-b8de0f928d54 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:00.392635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f261280-8e0e-41d2-b3c4-dbbfb1cb87f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:00.393474 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f261280-8e0e-41d2-b3c4-dbbfb1cb87f6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:00.393755 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f261280-8e0e-41d2-b3c4-dbbfb1cb87f6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:00.394030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f261280-8e0e-41d2-b3c4-dbbfb1cb87f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:00.407728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cfb63-30d4-4bf3-9207-7667c807f34f None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:00.409420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f261280-8e0e-41d2-b3c4-dbbfb1cb87f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-8f261280-8e0e-41d2-b3c4-dbbfb1cb87f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:00.425735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:00.426691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:00.427470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:00.427470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:00.427470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:00.427470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:00.427470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:00.427470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:00.427470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:00.427470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:00.427470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:00.427470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:00.427470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:13:00.427470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:00.427470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 212/424] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:13:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:00.434383 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f261280-8e0e-41d2-b3c4-dbbfb1cb87f6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:00.441834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f894ea-dbd3-41dc-be3b-de0b24f81f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:00.442719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f894ea-dbd3-41dc-be3b-de0b24f81f49 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:00.443052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f894ea-dbd3-41dc-be3b-de0b24f81f49 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:00.443332 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f894ea-dbd3-41dc-be3b-de0b24f81f49 None admin] PATH_INFO: `/v3/groups/39e8fddd854f443f986b1055833619da` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:00.444564 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f894ea-dbd3-41dc-be3b-de0b24f81f49 None admin] RBAC: Authorizing `identity:get_group(group_id=39e8fddd854f443f986b1055833619da)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:00.446090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f894ea-dbd3-41dc-be3b-de0b24f81f49 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:00.452016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 213/425] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:13:00 2022] GET /identity/v3/groups/39e8fddd854f443f986b1055833619da => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:00.454695 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f894ea-dbd3-41dc-be3b-de0b24f81f49 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:00.558836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f894ea-dbd3-41dc-be3b-de0b24f81f49 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:00.596595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce98d9-a607-4a42-a117-90fab7adb7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:00.597612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce98d9-a607-4a42-a117-90fab7adb7bd None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:00.598085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce98d9-a607-4a42-a117-90fab7adb7bd None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:00.598549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce98d9-a607-4a42-a117-90fab7adb7bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:00.599569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce98d9-a607-4a42-a117-90fab7adb7bd None admin] RBAC: Adding query filter params (group.id=39e8fddd854f443f986b1055833619da, role.id=d8b94102152a46768d568f66794854f3, scope.project.id=37e07551c481458b8e025b20f23fd735) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:00.600127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce98d9-a607-4a42-a117-90fab7adb7bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:00.601740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce98d9-a607-4a42-a117-90fab7adb7bd None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:00.609944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 214/426] 10.222.0.44 () {56 vars in 1376 bytes} [Mon May 9 04:13:00 2022] GET /identity/v3/role_assignments?group.id=39e8fddd854f443f986b1055833619da&scope.project.id=37e07551c481458b8e025b20f23fd735&role.id=d8b94102152a46768d568f66794854f3 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:00.611422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a92365-17dc-4a09-84f1-d0396468904f None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:00.611941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a92365-17dc-4a09-84f1-d0396468904f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:00.612451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a92365-17dc-4a09-84f1-d0396468904f None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:00.619123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 215/427] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:00.621440 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f261280-8e0e-41d2-b3c4-dbbfb1cb87f6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:00.625853 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c30f2-271e-43a9-afec-c6df26e2acd6 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:00.627274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c30f2-271e-43a9-afec-c6df26e2acd6 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:00.627771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c30f2-271e-43a9-afec-c6df26e2acd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:00.671657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727278f-ea7c-496d-9fd2-dc27ce9d8cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:00.672786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727278f-ea7c-496d-9fd2-dc27ce9d8cf3 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:00.673306 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727278f-ea7c-496d-9fd2-dc27ce9d8cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:00.673792 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727278f-ea7c-496d-9fd2-dc27ce9d8cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:00.674767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727278f-ea7c-496d-9fd2-dc27ce9d8cf3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:00.675293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727278f-ea7c-496d-9fd2-dc27ce9d8cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:00.676548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-be6c30f2-271e-43a9-afec-c6df26e2acd6 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:00.679530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-be6c30f2-271e-43a9-afec-c6df26e2acd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:00.680078 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727278f-ea7c-496d-9fd2-dc27ce9d8cf3 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:00.698138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 213/428] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:00.706246 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727278f-ea7c-496d-9fd2-dc27ce9d8cf3 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:00.822383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 216/429] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:00.852758 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c30f2-271e-43a9-afec-c6df26e2acd6 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:00.887724 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727278f-ea7c-496d-9fd2-dc27ce9d8cf3 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:00.932649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856f2c4-8e1e-4ebf-a666-cee7c0629fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:00.933300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856f2c4-8e1e-4ebf-a666-cee7c0629fba None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:00.933580 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856f2c4-8e1e-4ebf-a666-cee7c0629fba None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:00.933853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856f2c4-8e1e-4ebf-a666-cee7c0629fba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:00.976908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c30f2-271e-43a9-afec-c6df26e2acd6 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:01.011219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856f2c4-8e1e-4ebf-a666-cee7c0629fba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:01.012778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856f2c4-8e1e-4ebf-a666-cee7c0629fba None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:01.026307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd6bf3-56b0-467d-8cc6-49a3d411de2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:01.027568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd6bf3-56b0-467d-8cc6-49a3d411de2e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:01.028124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd6bf3-56b0-467d-8cc6-49a3d411de2e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:01.028937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd6bf3-56b0-467d-8cc6-49a3d411de2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:01.043232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd6bf3-56b0-467d-8cc6-49a3d411de2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-abfd6bf3-56b0-467d-8cc6-49a3d411de2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:01.052174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:01.053086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:01.053855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:01.053855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:01.053855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:01.053855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:01.053855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:01.053855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:01.053855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:01.053855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:01.053855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:01.053855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:01.053855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 04:13:01.053855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:01.058299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 217/430] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:01.064783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd6bf3-56b0-467d-8cc6-49a3d411de2e None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:01.119949 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 214/431] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:13:01.121504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856f2c4-8e1e-4ebf-a666-cee7c0629fba None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:01.170454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd6bf3-56b0-467d-8cc6-49a3d411de2e None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:01.208580 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0523dd-9bd5-42a7-9175-7a0856e0b6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:01.209648 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0523dd-9bd5-42a7-9175-7a0856e0b6ba None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:01.210127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0523dd-9bd5-42a7-9175-7a0856e0b6ba None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:01.210617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0523dd-9bd5-42a7-9175-7a0856e0b6ba None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:01.211636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0523dd-9bd5-42a7-9175-7a0856e0b6ba None admin] RBAC: Adding query filter params (name=member) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:01.212256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0523dd-9bd5-42a7-9175-7a0856e0b6ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:01.213941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0523dd-9bd5-42a7-9175-7a0856e0b6ba None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:01.224435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 218/432] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:13:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:01.227975 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856f2c4-8e1e-4ebf-a666-cee7c0629fba None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:01.230622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0523dd-9bd5-42a7-9175-7a0856e0b6ba None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:01.261060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9416d24-bf65-42ed-adac-1f9772542b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:01.261923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9416d24-bf65-42ed-adac-1f9772542b44 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:01.262211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9416d24-bf65-42ed-adac-1f9772542b44 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:01.262519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9416d24-bf65-42ed-adac-1f9772542b44 None admin] PATH_INFO: `/v3/projects/37e07551c481458b8e025b20f23fd735` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:01.263642 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9416d24-bf65-42ed-adac-1f9772542b44 None admin] RBAC: Authorizing `identity:get_project(project_id=37e07551c481458b8e025b20f23fd735)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:01.266085 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9416d24-bf65-42ed-adac-1f9772542b44 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:01.271790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 215/433] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:13:01 2022] GET /identity/v3/projects/37e07551c481458b8e025b20f23fd735 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:01.273495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9416d24-bf65-42ed-adac-1f9772542b44 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:01.333142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0523dd-9bd5-42a7-9175-7a0856e0b6ba None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:01.368053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b16b3-d4ab-4aca-9171-773a7bd21a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:01.369074 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b16b3-d4ab-4aca-9171-773a7bd21a5a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:01.369551 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b16b3-d4ab-4aca-9171-773a7bd21a5a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:01.370069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b16b3-d4ab-4aca-9171-773a7bd21a5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:01.372603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9416d24-bf65-42ed-adac-1f9772542b44 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:01.406268 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36456c-476d-4d43-9510-d40938ec28cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:01.407137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36456c-476d-4d43-9510-d40938ec28cd None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:01.407413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36456c-476d-4d43-9510-d40938ec28cd None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:01.407678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36456c-476d-4d43-9510-d40938ec28cd None admin] PATH_INFO: `/v3/groups/39e8fddd854f443f986b1055833619da` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:01.408899 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36456c-476d-4d43-9510-d40938ec28cd None admin] RBAC: Authorizing `identity:get_group(group_id=39e8fddd854f443f986b1055833619da)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:01.410341 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36456c-476d-4d43-9510-d40938ec28cd None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:01.414752 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 216/434] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:13:01 2022] GET /identity/v3/groups/39e8fddd854f443f986b1055833619da => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:01.416458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb6447-2924-4ab5-bce9-d7bb2f69ddbf None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:01.417043 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb6447-2924-4ab5-bce9-d7bb2f69ddbf None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:01.417517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb6447-2924-4ab5-bce9-d7bb2f69ddbf None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:01.419253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 217/435] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:01.420902 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb6447-2924-4ab5-bce9-d7bb2f69ddbf None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:01.446698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b16b3-d4ab-4aca-9171-773a7bd21a5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:01.448242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b16b3-d4ab-4aca-9171-773a7bd21a5a None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:01.562319 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb6447-2924-4ab5-bce9-d7bb2f69ddbf None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:01.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 219/436] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 04:13:01.570586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b997d6-3e22-4597-8449-48d754ed8bdb None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:01.570874 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b997d6-3e22-4597-8449-48d754ed8bdb None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:01.571152 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b997d6-3e22-4597-8449-48d754ed8bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:01.586850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-52b997d6-3e22-4597-8449-48d754ed8bdb None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:01.588032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-52b997d6-3e22-4597-8449-48d754ed8bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:01.596298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba78982-5ba1-4ffe-9fcc-d0f0d26a423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:01.596951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba78982-5ba1-4ffe-9fcc-d0f0d26a423d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:01.597230 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba78982-5ba1-4ffe-9fcc-d0f0d26a423d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:01.597503 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba78982-5ba1-4ffe-9fcc-d0f0d26a423d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:01.598317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba78982-5ba1-4ffe-9fcc-d0f0d26a423d None admin] RBAC: Adding query filter params (group.id=39e8fddd854f443f986b1055833619da, role.id=7600100dce614d9bb4d47b1e660b7bb2, scope.project.id=37e07551c481458b8e025b20f23fd735) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:01.598621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba78982-5ba1-4ffe-9fcc-d0f0d26a423d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:01.600058 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba78982-5ba1-4ffe-9fcc-d0f0d26a423d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:01.606208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 218/437] 10.222.0.44 () {56 vars in 1376 bytes} [Mon May 9 04:13:01 2022] GET /identity/v3/role_assignments?group.id=39e8fddd854f443f986b1055833619da&scope.project.id=37e07551c481458b8e025b20f23fd735&role.id=7600100dce614d9bb4d47b1e660b7bb2 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:01.607019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba78982-5ba1-4ffe-9fcc-d0f0d26a423d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:01.661667 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 220/438] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:01.662968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5ace7-093a-439b-b0e6-943f181e2ec2 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:01.663320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5ace7-093a-439b-b0e6-943f181e2ec2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:01.663597 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5ace7-093a-439b-b0e6-943f181e2ec2 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:01.665321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 221/439] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:01.674828 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c81775-9336-4116-ba9b-c9305cc18c60 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:01.675481 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c81775-9336-4116-ba9b-c9305cc18c60 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:01.676029 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c81775-9336-4116-ba9b-c9305cc18c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:01.695316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-58c81775-9336-4116-ba9b-c9305cc18c60 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:01.696631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-58c81775-9336-4116-ba9b-c9305cc18c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:01.739508 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba78982-5ba1-4ffe-9fcc-d0f0d26a423d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:01.771458 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 222/440] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:13:01.772271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c81775-9336-4116-ba9b-c9305cc18c60 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:01.787136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965be09-451c-46ff-999d-3d5931b23d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:01.788400 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965be09-451c-46ff-999d-3d5931b23d4f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:01.788929 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965be09-451c-46ff-999d-3d5931b23d4f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:01.789405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965be09-451c-46ff-999d-3d5931b23d4f None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:01.790834 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965be09-451c-46ff-999d-3d5931b23d4f None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:01.792481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965be09-451c-46ff-999d-3d5931b23d4f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:01.794367 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 219/441] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:13:01 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:01.798466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965be09-451c-46ff-999d-3d5931b23d4f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:01.894302 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c81775-9336-4116-ba9b-c9305cc18c60 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:01.913944 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965be09-451c-46ff-999d-3d5931b23d4f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:01.931127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22458a2c-43f7-42de-aa52-24104cfc6b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:01.931421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22458a2c-43f7-42de-aa52-24104cfc6b29 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:01.931704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22458a2c-43f7-42de-aa52-24104cfc6b29 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:01.931996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22458a2c-43f7-42de-aa52-24104cfc6b29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:01.944968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22458a2c-43f7-42de-aa52-24104cfc6b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:01.952940 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa6410-83dd-46d2-aaed-75a5213a12e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-22458a2c-43f7-42de-aa52-24104cfc6b29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:01.953127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:01.954426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:01.955416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:01.955416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:01.955416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:01.955416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:01.955416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:01.955416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:01.955416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:01.955416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:01.955416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:01.955416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:01.955416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:01.955416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:01.956286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa6410-83dd-46d2-aaed-75a5213a12e5 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:01.956286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa6410-83dd-46d2-aaed-75a5213a12e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:01.956286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa6410-83dd-46d2-aaed-75a5213a12e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:01.956499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 223/442] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:13:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:01.956589 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22458a2c-43f7-42de-aa52-24104cfc6b29 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:01.966481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa6410-83dd-46d2-aaed-75a5213a12e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-39fa6410-83dd-46d2-aaed-75a5213a12e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:01.976286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:01.977196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:01.977976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:01.977976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:01.977976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:01.977976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:01.977976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:01.977976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:01.977976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:01.977976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:01.977976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:01.977976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:01.977976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:13:01.977976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:01.977976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 220/443] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:13:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:01.978644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa6410-83dd-46d2-aaed-75a5213a12e5 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:02.064192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22458a2c-43f7-42de-aa52-24104cfc6b29 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:02.078935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa6410-83dd-46d2-aaed-75a5213a12e5 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:02.096110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef2f1f-c714-4cd5-a868-c73f4a51af6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:02.096963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef2f1f-c714-4cd5-a868-c73f4a51af6e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:02.097271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef2f1f-c714-4cd5-a868-c73f4a51af6e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:02.097552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef2f1f-c714-4cd5-a868-c73f4a51af6e None admin] PATH_INFO: `/v3/groups/39e8fddd854f443f986b1055833619da` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:02.098723 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef2f1f-c714-4cd5-a868-c73f4a51af6e None admin] RBAC: Authorizing `identity:get_group(group_id=39e8fddd854f443f986b1055833619da)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:02.100212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef2f1f-c714-4cd5-a868-c73f4a51af6e None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:02.101986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 224/444] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:13:01 2022] GET /identity/v3/groups/39e8fddd854f443f986b1055833619da => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:02.102562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef2f1f-c714-4cd5-a868-c73f4a51af6e None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:02.114816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb199032-cab9-4f19-b525-24796379c769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:02.115485 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb199032-cab9-4f19-b525-24796379c769 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:02.115767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb199032-cab9-4f19-b525-24796379c769 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:02.116084 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb199032-cab9-4f19-b525-24796379c769 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:02.116827 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb199032-cab9-4f19-b525-24796379c769 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:02.117135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb199032-cab9-4f19-b525-24796379c769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:02.118491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb199032-cab9-4f19-b525-24796379c769 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:02.126707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 221/445] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:02.128449 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb199032-cab9-4f19-b525-24796379c769 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:02.194181 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef2f1f-c714-4cd5-a868-c73f4a51af6e None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:02.220468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb199032-cab9-4f19-b525-24796379c769 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:02.228539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ad55f-1456-467b-8424-24498c465476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:02.229205 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ad55f-1456-467b-8424-24498c465476 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:02.229493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ad55f-1456-467b-8424-24498c465476 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:02.229767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ad55f-1456-467b-8424-24498c465476 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:02.230509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ad55f-1456-467b-8424-24498c465476 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:02.230813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ad55f-1456-467b-8424-24498c465476 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:02.232194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ad55f-1456-467b-8424-24498c465476 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:02.240027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 225/446] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:02.240807 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ad55f-1456-467b-8424-24498c465476 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:02.252175 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b574d5b-ef2a-4029-830d-5206cd7fbbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:02.252828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b574d5b-ef2a-4029-830d-5206cd7fbbac None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:02.253109 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b574d5b-ef2a-4029-830d-5206cd7fbbac None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:02.253384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b574d5b-ef2a-4029-830d-5206cd7fbbac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:02.254207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b574d5b-ef2a-4029-830d-5206cd7fbbac None admin] RBAC: Adding query filter params (group.id=39e8fddd854f443f986b1055833619da, role.id=d8b94102152a46768d568f66794854f3, scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:02.254520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b574d5b-ef2a-4029-830d-5206cd7fbbac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:02.255950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b574d5b-ef2a-4029-830d-5206cd7fbbac None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:02.261498 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 222/447] 10.222.0.44 () {56 vars in 1376 bytes} [Mon May 9 04:13:02 2022] GET /identity/v3/role_assignments?group.id=39e8fddd854f443f986b1055833619da&scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f&role.id=d8b94102152a46768d568f66794854f3 => generated 611 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:02.262261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b574d5b-ef2a-4029-830d-5206cd7fbbac None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:02.343381 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ad55f-1456-467b-8424-24498c465476 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:02.365426 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b574d5b-ef2a-4029-830d-5206cd7fbbac None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:02.378091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b039f2-2af5-4064-bbfd-1586abf7fa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:02.378753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b039f2-2af5-4064-bbfd-1586abf7fa5b None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:02.379035 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b039f2-2af5-4064-bbfd-1586abf7fa5b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:02.379308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b039f2-2af5-4064-bbfd-1586abf7fa5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:02.400531 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a72d08-436c-4a7d-afc4-ad03451351bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:02.401205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a72d08-436c-4a7d-afc4-ad03451351bb None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:02.401496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a72d08-436c-4a7d-afc4-ad03451351bb None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:02.401778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a72d08-436c-4a7d-afc4-ad03451351bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:02.452782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b039f2-2af5-4064-bbfd-1586abf7fa5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:02.454747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b039f2-2af5-4064-bbfd-1586abf7fa5b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:02.469427 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a72d08-436c-4a7d-afc4-ad03451351bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:02.471297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a72d08-436c-4a7d-afc4-ad03451351bb None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:02.558855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 226/448] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:13:02.565276 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b039f2-2af5-4064-bbfd-1586abf7fa5b None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:02.569133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 223/449] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:13:02.575319 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a72d08-436c-4a7d-afc4-ad03451351bb None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:02.670878 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b039f2-2af5-4064-bbfd-1586abf7fa5b None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:02.674716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a72d08-436c-4a7d-afc4-ad03451351bb None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:02.702683 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e7732-8743-436f-a3f8-3a683bab73fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:02.703843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e7732-8743-436f-a3f8-3a683bab73fc None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:02.704120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e7732-8743-436f-a3f8-3a683bab73fc None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:02.704392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e7732-8743-436f-a3f8-3a683bab73fc None admin] PATH_INFO: `/v3/projects/528420bd736843dc87d64954f4596bbd` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:02.705531 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e7732-8743-436f-a3f8-3a683bab73fc None admin] RBAC: Authorizing `identity:get_project(project_id=528420bd736843dc87d64954f4596bbd)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:02.707018 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e7732-8743-436f-a3f8-3a683bab73fc None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:02.707459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4110ee4-e0a9-4dfc-8b35-5297ddea23d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:02.708347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4110ee4-e0a9-4dfc-8b35-5297ddea23d4 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:02.708884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4110ee4-e0a9-4dfc-8b35-5297ddea23d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:02.708988 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 227/450] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:13:02 2022] GET /identity/v3/projects/528420bd736843dc87d64954f4596bbd => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:02.709069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4110ee4-e0a9-4dfc-8b35-5297ddea23d4 None admin] PATH_INFO: `/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:02.710074 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4110ee4-e0a9-4dfc-8b35-5297ddea23d4 None admin] RBAC: Authorizing `identity:get_project(project_id=f6ce8231a2464a4ba12fa0e9b4b6732f)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:02.711496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4110ee4-e0a9-4dfc-8b35-5297ddea23d4 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:02.713402 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 224/451] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:13:02 2022] GET /identity/v3/projects/f6ce8231a2464a4ba12fa0e9b4b6732f => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:02.714295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e7732-8743-436f-a3f8-3a683bab73fc None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:02.718811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4110ee4-e0a9-4dfc-8b35-5297ddea23d4 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:02.809845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4110ee4-e0a9-4dfc-8b35-5297ddea23d4 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:02.811394 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e7732-8743-436f-a3f8-3a683bab73fc None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:02.841855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1a877-63c1-4e97-9e4b-e85f7213ffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:02.843207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1a877-63c1-4e97-9e4b-e85f7213ffa9 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:02.843483 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1a877-63c1-4e97-9e4b-e85f7213ffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:02.843749 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1a877-63c1-4e97-9e4b-e85f7213ffa9 None admin] PATH_INFO: `/v3/groups/39e8fddd854f443f986b1055833619da` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:02.843855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858df6ff-9fda-49f1-b9c6-7268904a7d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:02.844629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858df6ff-9fda-49f1-b9c6-7268904a7d6c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:02.844912 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858df6ff-9fda-49f1-b9c6-7268904a7d6c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:02.844999 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1a877-63c1-4e97-9e4b-e85f7213ffa9 None admin] RBAC: Authorizing `identity:get_group(group_id=39e8fddd854f443f986b1055833619da)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:02.845188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858df6ff-9fda-49f1-b9c6-7268904a7d6c None admin] PATH_INFO: `/v3/groups/9134223cfe874829a524f380c32e8b44` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:02.846346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1a877-63c1-4e97-9e4b-e85f7213ffa9 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:02.846433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858df6ff-9fda-49f1-b9c6-7268904a7d6c None admin] RBAC: Authorizing `identity:get_group(group_id=9134223cfe874829a524f380c32e8b44)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:02.847781 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858df6ff-9fda-49f1-b9c6-7268904a7d6c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:02.848278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 225/452] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:13:02 2022] GET /identity/v3/groups/39e8fddd854f443f986b1055833619da => generated 226 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:02.851972 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 228/453] 10.222.0.44 () {56 vars in 1180 bytes} [Mon May 9 04:13:02 2022] GET /identity/v3/groups/9134223cfe874829a524f380c32e8b44 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:02.853529 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1a877-63c1-4e97-9e4b-e85f7213ffa9 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:02.854719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858df6ff-9fda-49f1-b9c6-7268904a7d6c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:02.944704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1a877-63c1-4e97-9e4b-e85f7213ffa9 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:02.945123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858df6ff-9fda-49f1-b9c6-7268904a7d6c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:02.976815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7e0f9-e2a8-4242-9477-40dbf526b80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:02.977122 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fd083-ef59-4f4b-93eb-e0801f5e9866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:02.977448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7e0f9-e2a8-4242-9477-40dbf526b80d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:02.977725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7e0f9-e2a8-4242-9477-40dbf526b80d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:02.977814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fd083-ef59-4f4b-93eb-e0801f5e9866 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:02.977994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7e0f9-e2a8-4242-9477-40dbf526b80d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:02.978094 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fd083-ef59-4f4b-93eb-e0801f5e9866 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:02.978299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fd083-ef59-4f4b-93eb-e0801f5e9866 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:02.978815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7e0f9-e2a8-4242-9477-40dbf526b80d None admin] RBAC: Adding query filter params (group.id=9134223cfe874829a524f380c32e8b44, role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=528420bd736843dc87d64954f4596bbd) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:02.979114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fd083-ef59-4f4b-93eb-e0801f5e9866 None admin] RBAC: Adding query filter params (group.id=39e8fddd854f443f986b1055833619da, role.id=7600100dce614d9bb4d47b1e660b7bb2, scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:02.979202 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7e0f9-e2a8-4242-9477-40dbf526b80d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:02.979425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fd083-ef59-4f4b-93eb-e0801f5e9866 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:02.980542 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7e0f9-e2a8-4242-9477-40dbf526b80d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:02.980844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fd083-ef59-4f4b-93eb-e0801f5e9866 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:02.986570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 226/454] 10.222.0.44 () {56 vars in 1376 bytes} [Mon May 9 04:13:02 2022] GET /identity/v3/role_assignments?group.id=39e8fddd854f443f986b1055833619da&scope.project.id=f6ce8231a2464a4ba12fa0e9b4b6732f&role.id=7600100dce614d9bb4d47b1e660b7bb2 => generated 611 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:02.995795 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 229/455] 10.222.0.44 () {56 vars in 1376 bytes} [Mon May 9 04:13:02 2022] GET /identity/v3/role_assignments?group.id=9134223cfe874829a524f380c32e8b44&scope.project.id=528420bd736843dc87d64954f4596bbd&role.id=e59113c0366049e889df3f55eed90c72 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:05.788572 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37136a-d8dd-4e17-b9d0-63189fcb0172 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:05.789553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37136a-d8dd-4e17-b9d0-63189fcb0172 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:05.790137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37136a-d8dd-4e17-b9d0-63189fcb0172 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:05.791989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 227/456] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:05.797900 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6dc74-72e6-416d-a9d9-45665afdaf3e None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:05.798318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6dc74-72e6-416d-a9d9-45665afdaf3e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:05.798505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6dc74-72e6-416d-a9d9-45665afdaf3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:05.822622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-15a6dc74-72e6-416d-a9d9-45665afdaf3e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:05.823800 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-15a6dc74-72e6-416d-a9d9-45665afdaf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:05.918932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 230/457] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:05.960851 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37136a-d8dd-4e17-b9d0-63189fcb0172 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:06.129662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37136a-d8dd-4e17-b9d0-63189fcb0172 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:06.179714 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da8a51-39e2-470a-84c9-f7bf9dda2ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:06.180575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da8a51-39e2-470a-84c9-f7bf9dda2ecd None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:06.180860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da8a51-39e2-470a-84c9-f7bf9dda2ecd None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:06.181139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da8a51-39e2-470a-84c9-f7bf9dda2ecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:06.193490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da8a51-39e2-470a-84c9-f7bf9dda2ecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:06.194942 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da8a51-39e2-470a-84c9-f7bf9dda2ecd None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-e5da8a51-39e2-470a-84c9-f7bf9dda2ecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:06.221172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:06.222086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:06.222930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:06.222930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:06.222930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:06.222930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:06.223147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc38af-ae26-4d29-ade0-cf28fe203364 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:06.223147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc38af-ae26-4d29-ade0-cf28fe203364 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:06.223147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc38af-ae26-4d29-ade0-cf28fe203364 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:06.223334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 228/458] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:06.223865 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 231/459] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:06.229141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da8a51-39e2-470a-84c9-f7bf9dda2ecd None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:06.229909 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f287a8-afd5-4318-951a-1ac6d61f7406 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:06.230202 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f287a8-afd5-4318-951a-1ac6d61f7406 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:06.230485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f287a8-afd5-4318-951a-1ac6d61f7406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:06.251161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-b3f287a8-afd5-4318-951a-1ac6d61f7406 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:06.252393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b3f287a8-afd5-4318-951a-1ac6d61f7406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:06.339084 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 232/460] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:13:06.363678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da8a51-39e2-470a-84c9-f7bf9dda2ecd None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:06.382288 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee438906-7434-49d9-acb4-ab5b2e77d07d None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:06.382561 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee438906-7434-49d9-acb4-ab5b2e77d07d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:06.382782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee438906-7434-49d9-acb4-ab5b2e77d07d None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:06.384419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 233/461] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:13:06.392451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee438906-7434-49d9-acb4-ab5b2e77d07d None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:06.397311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1837b-b652-4752-9f7e-836b1cac758d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:06.397976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1837b-b652-4752-9f7e-836b1cac758d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:06.398253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1837b-b652-4752-9f7e-836b1cac758d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:06.398525 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1837b-b652-4752-9f7e-836b1cac758d None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:06.399247 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1837b-b652-4752-9f7e-836b1cac758d None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:06.399546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1837b-b652-4752-9f7e-836b1cac758d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:06.400932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1837b-b652-4752-9f7e-836b1cac758d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:06.412596 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 229/462] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:06.413843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b5292-648c-4058-906b-1fa6b17d4bd5 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:06.414128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b5292-648c-4058-906b-1fa6b17d4bd5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:06.414402 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b5292-648c-4058-906b-1fa6b17d4bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:06.433832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-4f4b5292-648c-4058-906b-1fa6b17d4bd5 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:06.435004 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-4f4b5292-648c-4058-906b-1fa6b17d4bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:06.504739 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 230/463] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:06.505579 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b5292-648c-4058-906b-1fa6b17d4bd5 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:06.518951 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee438906-7434-49d9-acb4-ab5b2e77d07d None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:06.570908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ef230-7182-4894-9c1b-65edd4706206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:06.572471 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef230-7182-4894-9c1b-65edd4706206 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:06.572759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef230-7182-4894-9c1b-65edd4706206 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:06.573037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef230-7182-4894-9c1b-65edd4706206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:06.586298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ef230-7182-4894-9c1b-65edd4706206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:06.588374 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ef230-7182-4894-9c1b-65edd4706206 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:06.608429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b5292-648c-4058-906b-1fa6b17d4bd5 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-e04ef230-7182-4894-9c1b-65edd4706206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:06.613022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:06.614409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:06.616443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:06.616443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:06.616443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:06.616443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:06.619492 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 234/464] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:06.620397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ef230-7182-4894-9c1b-65edd4706206 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:06.646826 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85973aa-9064-4798-ad7e-9cdb38617b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:06.647568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85973aa-9064-4798-ad7e-9cdb38617b8c None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:06.647982 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85973aa-9064-4798-ad7e-9cdb38617b8c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:06.648376 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85973aa-9064-4798-ad7e-9cdb38617b8c None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:06.649352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85973aa-9064-4798-ad7e-9cdb38617b8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:06.650898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85973aa-9064-4798-ad7e-9cdb38617b8c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:06.672168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 231/465] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:13:06 2022] POST /identity/v3/users => generated 246 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:13:06.672986 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85973aa-9064-4798-ad7e-9cdb38617b8c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:06.754313 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ef230-7182-4894-9c1b-65edd4706206 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:06.798293 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd272a-25c2-458f-a7d5-23d5e03c0535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:06.799094 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd272a-25c2-458f-a7d5-23d5e03c0535 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:06.799376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd272a-25c2-458f-a7d5-23d5e03c0535 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:06.799654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd272a-25c2-458f-a7d5-23d5e03c0535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:06.803179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85973aa-9064-4798-ad7e-9cdb38617b8c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:06.812238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd272a-25c2-458f-a7d5-23d5e03c0535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:06.813785 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd272a-25c2-458f-a7d5-23d5e03c0535 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-38bd272a-25c2-458f-a7d5-23d5e03c0535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:06.833198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:06.834116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:06.835390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:06.835390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:06.835390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:06.835390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:06.835390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 235/466] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:06.836155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff98ed-c8b3-4e24-b32e-ecff25e0b01b None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:06.836440 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff98ed-c8b3-4e24-b32e-ecff25e0b01b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:06.836710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff98ed-c8b3-4e24-b32e-ecff25e0b01b None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:06.842039 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 236/467] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:06.842039 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff98ed-c8b3-4e24-b32e-ecff25e0b01b None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:06.849657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3d55f-e9bb-492f-88bc-7f4eeb9061ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:06.850673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3d55f-e9bb-492f-88bc-7f4eeb9061ff None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:06.851171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3d55f-e9bb-492f-88bc-7f4eeb9061ff None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:06.851637 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3d55f-e9bb-492f-88bc-7f4eeb9061ff None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:06.852644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3d55f-e9bb-492f-88bc-7f4eeb9061ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:06.853181 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3d55f-e9bb-492f-88bc-7f4eeb9061ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:06.854798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3d55f-e9bb-492f-88bc-7f4eeb9061ff None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:06.867970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 232/468] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:06.869753 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a0ba0-240e-4ca7-a67b-e0d9bf875b26 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:06.870239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a0ba0-240e-4ca7-a67b-e0d9bf875b26 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:06.870725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a0ba0-240e-4ca7-a67b-e0d9bf875b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:06.889766 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f45a0ba0-240e-4ca7-a67b-e0d9bf875b26 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:06.891334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f45a0ba0-240e-4ca7-a67b-e0d9bf875b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:06.957990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff98ed-c8b3-4e24-b32e-ecff25e0b01b None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:06.962696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 233/469] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:06.963540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a0ba0-240e-4ca7-a67b-e0d9bf875b26 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:06.991169 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c08393-e7eb-4ebc-8158-6aecfea9cd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:06.991844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c08393-e7eb-4ebc-8158-6aecfea9cd0f None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:06.992159 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c08393-e7eb-4ebc-8158-6aecfea9cd0f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:06.992439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c08393-e7eb-4ebc-8158-6aecfea9cd0f None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:06.993231 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c08393-e7eb-4ebc-8158-6aecfea9cd0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:06.993545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c08393-e7eb-4ebc-8158-6aecfea9cd0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:06.994928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c08393-e7eb-4ebc-8158-6aecfea9cd0f None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:07.012707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 237/470] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:07.013529 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c08393-e7eb-4ebc-8158-6aecfea9cd0f None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:07.080682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a0ba0-240e-4ca7-a67b-e0d9bf875b26 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:07.115424 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9ed1d-f8c5-4f6f-aee5-0614e78a7386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:07.116416 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9ed1d-f8c5-4f6f-aee5-0614e78a7386 None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:07.116870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9ed1d-f8c5-4f6f-aee5-0614e78a7386 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:07.117312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9ed1d-f8c5-4f6f-aee5-0614e78a7386 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:07.118197 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9ed1d-f8c5-4f6f-aee5-0614e78a7386 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:07.119885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9ed1d-f8c5-4f6f-aee5-0614e78a7386 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:07.142014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c08393-e7eb-4ebc-8158-6aecfea9cd0f None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:07.143164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 234/471] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:13:06 2022] POST /identity/v3/users => generated 248 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:13:07.144367 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9ed1d-f8c5-4f6f-aee5-0614e78a7386 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:07.180079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08554c66-046f-4a23-b633-be4f963fa14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:07.180984 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08554c66-046f-4a23-b633-be4f963fa14e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:07.181330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08554c66-046f-4a23-b633-be4f963fa14e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:07.181730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08554c66-046f-4a23-b633-be4f963fa14e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:07.193503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08554c66-046f-4a23-b633-be4f963fa14e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:07.195192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08554c66-046f-4a23-b633-be4f963fa14e None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-08554c66-046f-4a23-b633-be4f963fa14e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:07.215104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:07.216057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:07.217529 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:07.217529 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:07.217529 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:07.217529 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:07.220197 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 238/472] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:07.223609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08554c66-046f-4a23-b633-be4f963fa14e None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:07.269105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9ed1d-f8c5-4f6f-aee5-0614e78a7386 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:07.305827 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9234ba-6d20-4c3a-ba07-99843373f8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:07.306614 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9234ba-6d20-4c3a-ba07-99843373f8dd None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:07.307142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9234ba-6d20-4c3a-ba07-99843373f8dd None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:07.307249 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9234ba-6d20-4c3a-ba07-99843373f8dd None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:07.308106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9234ba-6d20-4c3a-ba07-99843373f8dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:07.309677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9234ba-6d20-4c3a-ba07-99843373f8dd None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:07.337857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 235/473] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:13:07 2022] POST /identity/v3/users => generated 248 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:13:07.339578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8d4c5-21a7-4811-a9b4-2316bd605e35 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:07.340708 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8d4c5-21a7-4811-a9b4-2316bd605e35 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:07.341188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8d4c5-21a7-4811-a9b4-2316bd605e35 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:07.344318 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 236/474] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:07.349827 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2ebb7-cbcc-4c98-914d-16b297ce63e2 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:07.350128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2ebb7-cbcc-4c98-914d-16b297ce63e2 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:07.350389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2ebb7-cbcc-4c98-914d-16b297ce63e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:07.359473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08554c66-046f-4a23-b633-be4f963fa14e None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:07.369311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-26d2ebb7-cbcc-4c98-914d-16b297ce63e2 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:07.370612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-26d2ebb7-cbcc-4c98-914d-16b297ce63e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:07.401111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739c2e9-4d44-4fdf-a30b-7c092133a63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:07.402193 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739c2e9-4d44-4fdf-a30b-7c092133a63a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:07.402670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739c2e9-4d44-4fdf-a30b-7c092133a63a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:07.403136 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739c2e9-4d44-4fdf-a30b-7c092133a63a None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:07.404247 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739c2e9-4d44-4fdf-a30b-7c092133a63a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:07.404811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739c2e9-4d44-4fdf-a30b-7c092133a63a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:07.406415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739c2e9-4d44-4fdf-a30b-7c092133a63a None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:07.423658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 239/475] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:07.425075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739c2e9-4d44-4fdf-a30b-7c092133a63a None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:07.453911 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 237/476] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:13:07.479064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2ebb7-cbcc-4c98-914d-16b297ce63e2 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:07.534344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739c2e9-4d44-4fdf-a30b-7c092133a63a None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:07.571451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d215526-c640-4ac1-9b0b-b3fce25896f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:07.572706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d215526-c640-4ac1-9b0b-b3fce25896f6 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:07.573528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d215526-c640-4ac1-9b0b-b3fce25896f6 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:07.574042 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d215526-c640-4ac1-9b0b-b3fce25896f6 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:07.574993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d215526-c640-4ac1-9b0b-b3fce25896f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:07.578094 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d215526-c640-4ac1-9b0b-b3fce25896f6 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:07.590196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2ebb7-cbcc-4c98-914d-16b297ce63e2 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:07.612641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 240/477] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:13:07 2022] POST /identity/v3/users => generated 249 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:13:07.646035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0d296-361e-4e17-8ee7-95241bf3f761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:07.646751 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0d296-361e-4e17-8ee7-95241bf3f761 None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:07.647033 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0d296-361e-4e17-8ee7-95241bf3f761 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:07.647309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0d296-361e-4e17-8ee7-95241bf3f761 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:07.648102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b0d296-361e-4e17-8ee7-95241bf3f761 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:07.649816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b0d296-361e-4e17-8ee7-95241bf3f761 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:07.657476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.sql.core [None req-f9b0d296-361e-4e17-8ee7-95241bf3f761 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 04:13:07.657476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 04:13:07.657476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [parameters: {'id': '41affd2c1c9d4d78949e704059a8f942', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 04:13:07.657476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79763) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-f9b0d296-361e-4e17-8ee7-95241bf3f761 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application result = self._query(query) May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application conn.query(q) May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application result.read() May 09 04:13:07.678002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return ref.to_dict() May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 04:13:07.679105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application next(self.gen) May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application yield resource May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application next(self.gen) May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self.session.rollback() May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self.force_reraise() May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise self.value May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application session.commit() May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 04:13:07.680408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._prepare_impl() May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self.session.flush() May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._flush(objects) May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application compat.raise_( May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application flush_context.execute() May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rec.execute(self) May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 04:13:07.681239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 04:13:07.682102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application result = self._query(query) May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application conn.query(q) May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application result.read() May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 04:13:07.682903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [parameters: {'id': '41affd2c1c9d4d78949e704059a8f942', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:07.683719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 04:13:07.690356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:07.692321 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 238/478] 10.222.0.44 () {60 vars in 1131 bytes} [Mon May 9 04:13:07 2022] POST /identity/v3/roles => generated 144 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:07.692452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d215526-c640-4ac1-9b0b-b3fce25896f6 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:07.808809 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d215526-c640-4ac1-9b0b-b3fce25896f6 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:07.847866 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7fb39-aeeb-4537-bcb7-915fe3e4d28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:07.848704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7fb39-aeeb-4537-bcb7-915fe3e4d28a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:07.848987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7fb39-aeeb-4537-bcb7-915fe3e4d28a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:07.849352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7fb39-aeeb-4537-bcb7-915fe3e4d28a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:07.862884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7fb39-aeeb-4537-bcb7-915fe3e4d28a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-6cf7fb39-aeeb-4537-bcb7-915fe3e4d28a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:07.871014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:07.871925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:07.872738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:07.872738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:07.872738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:07.872738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:07.872738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:07.872738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:07.872738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:07.872738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:07.872738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:07.872738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:07.872738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 04:13:07.872738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:07.872738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 241/479] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:13:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:07.879586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0d296-361e-4e17-8ee7-95241bf3f761 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:08.000738 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0d296-361e-4e17-8ee7-95241bf3f761 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:08.047333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09568a8-cc82-49b5-ba3b-c99ebc2b3eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:08.048313 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09568a8-cc82-49b5-ba3b-c99ebc2b3eed None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:08.048758 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09568a8-cc82-49b5-ba3b-c99ebc2b3eed None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:08.049203 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09568a8-cc82-49b5-ba3b-c99ebc2b3eed None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:08.050097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09568a8-cc82-49b5-ba3b-c99ebc2b3eed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:08.050557 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09568a8-cc82-49b5-ba3b-c99ebc2b3eed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:08.052086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09568a8-cc82-49b5-ba3b-c99ebc2b3eed None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:08.067337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 239/480] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:09.553982 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c577846-776a-4908-9a0c-feb9eac9b2cf None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:09.554838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c577846-776a-4908-9a0c-feb9eac9b2cf None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:09.555292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c577846-776a-4908-9a0c-feb9eac9b2cf None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:09.557102 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 242/481] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:09.563155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ca231-d6e5-4320-90d4-af5771264483 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:09.563919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ca231-d6e5-4320-90d4-af5771264483 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:09.564345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ca231-d6e5-4320-90d4-af5771264483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:09.584673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f84ca231-d6e5-4320-90d4-af5771264483 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:09.586133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f84ca231-d6e5-4320-90d4-af5771264483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:09.678297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 240/482] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:09.721330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c577846-776a-4908-9a0c-feb9eac9b2cf None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:09.841096 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c577846-776a-4908-9a0c-feb9eac9b2cf None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:09.879726 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689dda0-758a-4f35-a76e-276214210bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:09.880804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689dda0-758a-4f35-a76e-276214210bdd None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:09.881231 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689dda0-758a-4f35-a76e-276214210bdd None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:09.881648 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689dda0-758a-4f35-a76e-276214210bdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:09.893941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689dda0-758a-4f35-a76e-276214210bdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-2689dda0-758a-4f35-a76e-276214210bdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:09.902285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:09.903398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:09.904430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:09.904430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:09.904430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:09.904430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:09.904430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:09.904430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:09.904430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:09.904430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:09.904430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:09.904430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:09.904430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:09.904430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:09.906888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 243/483] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:09.912332 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ca231-d6e5-4320-90d4-af5771264483 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:10.017967 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ca231-d6e5-4320-90d4-af5771264483 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:10.059407 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15de6cd-a1ec-44ed-a6fb-c6c9fe2ce145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:10.060111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15de6cd-a1ec-44ed-a6fb-c6c9fe2ce145 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.060918 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15de6cd-a1ec-44ed-a6fb-c6c9fe2ce145 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.061257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15de6cd-a1ec-44ed-a6fb-c6c9fe2ce145 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.062016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15de6cd-a1ec-44ed-a6fb-c6c9fe2ce145 None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:10.062316 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15de6cd-a1ec-44ed-a6fb-c6c9fe2ce145 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:10.063675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15de6cd-a1ec-44ed-a6fb-c6c9fe2ce145 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:10.072398 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 241/484] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:10.078796 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689dda0-758a-4f35-a76e-276214210bdd None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:10.124561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74d35f-3ccb-4f51-81f1-57653d713b07 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.124848 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74d35f-3ccb-4f51-81f1-57653d713b07 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.125040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74d35f-3ccb-4f51-81f1-57653d713b07 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.126434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 242/485] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:10.140564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14b5ff-ed0d-4858-a16b-3fd99fe53fc1 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.140903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14b5ff-ed0d-4858-a16b-3fd99fe53fc1 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.141131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14b5ff-ed0d-4858-a16b-3fd99fe53fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.169214 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-9a14b5ff-ed0d-4858-a16b-3fd99fe53fc1 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:10.171027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9a14b5ff-ed0d-4858-a16b-3fd99fe53fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:10.275432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689dda0-758a-4f35-a76e-276214210bdd None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:10.278568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 243/486] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:13:10.296128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14b5ff-ed0d-4858-a16b-3fd99fe53fc1 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:10.345994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3030a-2ee3-40e6-90ff-55bbb7897c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:10.346801 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3030a-2ee3-40e6-90ff-55bbb7897c43 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.347078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3030a-2ee3-40e6-90ff-55bbb7897c43 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.347350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3030a-2ee3-40e6-90ff-55bbb7897c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.364712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3030a-2ee3-40e6-90ff-55bbb7897c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:10.366704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3030a-2ee3-40e6-90ff-55bbb7897c43 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-b6d3030a-2ee3-40e6-90ff-55bbb7897c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:10.385490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:10.386396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:10.387253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:10.387253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:10.387253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:10.387253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:10.390530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 244/487] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:10.396970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf6bb9-ebc4-486b-9d1b-2a27333acbff None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.397634 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf6bb9-ebc4-486b-9d1b-2a27333acbff None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.398112 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf6bb9-ebc4-486b-9d1b-2a27333acbff None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.399829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 245/488] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:10.403995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14b5ff-ed0d-4858-a16b-3fd99fe53fc1 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:10.406071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf6bb9-ebc4-486b-9d1b-2a27333acbff None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:10.437602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ce052-fc86-4764-af81-83ca8935310f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:10.438359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ce052-fc86-4764-af81-83ca8935310f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.438637 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ce052-fc86-4764-af81-83ca8935310f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.438906 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ce052-fc86-4764-af81-83ca8935310f None admin] PATH_INFO: `/v3/roles/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.449521 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ce052-fc86-4764-af81-83ca8935310f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-c98ce052-fc86-4764-af81-83ca8935310f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:10.458343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:10.459317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:10.460127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:10.460127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:10.460127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:10.460127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:10.460127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:10.460127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:10.460127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:10.460127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:10.460127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:10.460127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:10.460127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:10.460127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:10.464051 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 244/489] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:10.465331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd992b7b-bfec-42cc-8f1e-9cdfa2bfef17 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.465616 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd992b7b-bfec-42cc-8f1e-9cdfa2bfef17 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.465893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd992b7b-bfec-42cc-8f1e-9cdfa2bfef17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.483856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-bd992b7b-bfec-42cc-8f1e-9cdfa2bfef17 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:10.485054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-bd992b7b-bfec-42cc-8f1e-9cdfa2bfef17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:10.518786 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf6bb9-ebc4-486b-9d1b-2a27333acbff None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:10.553364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5bc25-286a-4ef7-b3d1-78642a0c9536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:10.554411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5bc25-286a-4ef7-b3d1-78642a0c9536 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.559924 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 245/490] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:10.560076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5bc25-286a-4ef7-b3d1-78642a0c9536 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.560153 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd992b7b-bfec-42cc-8f1e-9cdfa2bfef17 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:10.561363 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5bc25-286a-4ef7-b3d1-78642a0c9536 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.562328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5bc25-286a-4ef7-b3d1-78642a0c9536 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:10.562845 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5bc25-286a-4ef7-b3d1-78642a0c9536 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:10.564475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5bc25-286a-4ef7-b3d1-78642a0c9536 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:10.593291 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 246/491] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:10.598961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5bc25-286a-4ef7-b3d1-78642a0c9536 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:10.659132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd992b7b-bfec-42cc-8f1e-9cdfa2bfef17 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:10.694326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0963ea1-d39d-4936-a32d-db39e2b9563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:10.695003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0963ea1-d39d-4936-a32d-db39e2b9563a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.695275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0963ea1-d39d-4936-a32d-db39e2b9563a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.695543 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0963ea1-d39d-4936-a32d-db39e2b9563a None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.696312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0963ea1-d39d-4936-a32d-db39e2b9563a None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:10.696613 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0963ea1-d39d-4936-a32d-db39e2b9563a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:10.697983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0963ea1-d39d-4936-a32d-db39e2b9563a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:10.704742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5bc25-286a-4ef7-b3d1-78642a0c9536 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:10.707469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 246/492] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:10.712594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0963ea1-d39d-4936-a32d-db39e2b9563a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:10.744761 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c4727-8d3f-4bae-a69e-11cf2a094463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:10.746232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c4727-8d3f-4bae-a69e-11cf2a094463 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.746716 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c4727-8d3f-4bae-a69e-11cf2a094463 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.747338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c4727-8d3f-4bae-a69e-11cf2a094463 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.758568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c4727-8d3f-4bae-a69e-11cf2a094463 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:10.760037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c4727-8d3f-4bae-a69e-11cf2a094463 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-c29c4727-8d3f-4bae-a69e-11cf2a094463 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:10.777075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:10.778014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:10.778921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:10.778921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:10.778921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:10.778921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:10.778921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:10.778921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:10.778921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:10.778921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:10.778921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 04:13:10.778921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:10.778921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 247/493] 10.222.0.44 () {56 vars in 1093 bytes} [Mon May 9 04:13:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:10.779699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e26a91-3491-4b3d-8bcc-07d02fed1431 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.780638 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e26a91-3491-4b3d-8bcc-07d02fed1431 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.780638 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e26a91-3491-4b3d-8bcc-07d02fed1431 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.783975 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 248/494] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:10.783975 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e26a91-3491-4b3d-8bcc-07d02fed1431 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:10.828319 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0963ea1-d39d-4936-a32d-db39e2b9563a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:10.865657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0182ce-b33f-4ace-a6b7-b2bd3722f3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:10.866464 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0182ce-b33f-4ace-a6b7-b2bd3722f3b6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.866746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0182ce-b33f-4ace-a6b7-b2bd3722f3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.867023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0182ce-b33f-4ace-a6b7-b2bd3722f3b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.886546 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e26a91-3491-4b3d-8bcc-07d02fed1431 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:10.890457 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0182ce-b33f-4ace-a6b7-b2bd3722f3b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-2c0182ce-b33f-4ace-a6b7-b2bd3722f3b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:10.898257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:10.899137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:10.899972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:10.899972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:10.899972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:10.899972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:10.899972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:10.899972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:10.899972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:10.899972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:10.899972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:10.899972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:10.899972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:10.899972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:10.899972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 247/495] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:10.900629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0182ce-b33f-4ace-a6b7-b2bd3722f3b6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:10.927076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e9adb-1176-4d60-8a5c-7b5aebb7e59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:10.927850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e9adb-1176-4d60-8a5c-7b5aebb7e59e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.928171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e9adb-1176-4d60-8a5c-7b5aebb7e59e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.928444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e9adb-1176-4d60-8a5c-7b5aebb7e59e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.940649 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e9adb-1176-4d60-8a5c-7b5aebb7e59e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:10.942127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e9adb-1176-4d60-8a5c-7b5aebb7e59e None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-114e9adb-1176-4d60-8a5c-7b5aebb7e59e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:10.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:10.966784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:10.967657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:10.967657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:10.967657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:10.967657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:10.970051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 249/496] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:10.971550 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d4fc1-b4ab-4dee-99be-21d06a7fae24 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:10.971550 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d4fc1-b4ab-4dee-99be-21d06a7fae24 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:10.971550 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d4fc1-b4ab-4dee-99be-21d06a7fae24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:10.989463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e33d4fc1-b4ab-4dee-99be-21d06a7fae24 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:10.990668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e33d4fc1-b4ab-4dee-99be-21d06a7fae24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:11.028560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0182ce-b33f-4ace-a6b7-b2bd3722f3b6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:11.063582 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 250/497] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:11.064225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7cd63-dfef-4dfd-8a7b-32572bb52f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:11.064442 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d4fc1-b4ab-4dee-99be-21d06a7fae24 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:11.065299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7cd63-dfef-4dfd-8a7b-32572bb52f66 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:11.065781 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7cd63-dfef-4dfd-8a7b-32572bb52f66 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:11.066270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7cd63-dfef-4dfd-8a7b-32572bb52f66 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:11.067331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7cd63-dfef-4dfd-8a7b-32572bb52f66 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:11.067710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7cd63-dfef-4dfd-8a7b-32572bb52f66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:11.069301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7cd63-dfef-4dfd-8a7b-32572bb52f66 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:11.108447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 248/498] 10.222.0.44 () {56 vars in 1133 bytes} [Mon May 9 04:13:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:11.109447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7cd63-dfef-4dfd-8a7b-32572bb52f66 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:11.165434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d4fc1-b4ab-4dee-99be-21d06a7fae24 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:11.198519 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba05655-bede-4e07-9e51-a27411978e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:11.199222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba05655-bede-4e07-9e51-a27411978e5e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:11.199513 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba05655-bede-4e07-9e51-a27411978e5e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:11.199794 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba05655-bede-4e07-9e51-a27411978e5e None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:11.200575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba05655-bede-4e07-9e51-a27411978e5e None admin] RBAC: Adding query filter params (name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:11.200883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba05655-bede-4e07-9e51-a27411978e5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:11.202349 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba05655-bede-4e07-9e51-a27411978e5e None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:11.206705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7cd63-dfef-4dfd-8a7b-32572bb52f66 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:11.210538 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 251/499] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:11.211318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba05655-bede-4e07-9e51-a27411978e5e None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:11.239553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba86a5-25fc-43ad-ba39-5c1cdd593f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:11.240256 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba86a5-25fc-43ad-ba39-5c1cdd593f52 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:11.240538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba86a5-25fc-43ad-ba39-5c1cdd593f52 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:11.240811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba86a5-25fc-43ad-ba39-5c1cdd593f52 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:11.241523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba86a5-25fc-43ad-ba39-5c1cdd593f52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:11.241823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba86a5-25fc-43ad-ba39-5c1cdd593f52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:11.243211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba86a5-25fc-43ad-ba39-5c1cdd593f52 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:11.253841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 249/500] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:11.254619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba86a5-25fc-43ad-ba39-5c1cdd593f52 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:11.308270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba05655-bede-4e07-9e51-a27411978e5e None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:11.340104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab1c3d-3d31-40b0-9768-161cc14b5830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:11.340887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab1c3d-3d31-40b0-9768-161cc14b5830 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:11.341166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab1c3d-3d31-40b0-9768-161cc14b5830 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:11.341438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab1c3d-3d31-40b0-9768-161cc14b5830 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:11.347864 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba86a5-25fc-43ad-ba39-5c1cdd593f52 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:11.351031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab1c3d-3d31-40b0-9768-161cc14b5830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-6eab1c3d-3d31-40b0-9768-161cc14b5830 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:11.358881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:11.359768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:11.360598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:11.360598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:11.360598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:11.360598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:11.360598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:11.360598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:11.360598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:11.360598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:11.360598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:11.360598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:11.360598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:11.360598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:11.360598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 252/501] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:11.361490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617e9d0-ebef-431e-8e3f-e6f29b189121 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:11.361768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617e9d0-ebef-431e-8e3f-e6f29b189121 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:11.362048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617e9d0-ebef-431e-8e3f-e6f29b189121 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:11.363286 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 253/502] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:11.365478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617e9d0-ebef-431e-8e3f-e6f29b189121 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:11.384765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b903b-fe5f-4d6a-ac1b-ae6527500c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:11.385415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b903b-fe5f-4d6a-ac1b-ae6527500c0b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:11.385696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b903b-fe5f-4d6a-ac1b-ae6527500c0b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:11.385970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b903b-fe5f-4d6a-ac1b-ae6527500c0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:11.449456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b903b-fe5f-4d6a-ac1b-ae6527500c0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:11.450826 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b903b-fe5f-4d6a-ac1b-ae6527500c0b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:11.460960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617e9d0-ebef-431e-8e3f-e6f29b189121 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:11.491442 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706948c-f669-432a-bf2f-a08eea747644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:11.492251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706948c-f669-432a-bf2f-a08eea747644 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:11.492530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706948c-f669-432a-bf2f-a08eea747644 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:11.492813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706948c-f669-432a-bf2f-a08eea747644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:11.504016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706948c-f669-432a-bf2f-a08eea747644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:11.505461 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706948c-f669-432a-bf2f-a08eea747644 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-c706948c-f669-432a-bf2f-a08eea747644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:11.522101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:11.522953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:11.523767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:11.523767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:11.523767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:11.523767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:11.523767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 254/503] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:11.524206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706948c-f669-432a-bf2f-a08eea747644 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:11.552283 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 250/504] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:13:11.552283 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b903b-fe5f-4d6a-ac1b-ae6527500c0b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:11.615719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706948c-f669-432a-bf2f-a08eea747644 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:11.641089 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b903b-fe5f-4d6a-ac1b-ae6527500c0b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:11.646415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab553f44-677c-4506-b2ae-0161c69b3941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:11.647197 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab553f44-677c-4506-b2ae-0161c69b3941 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:11.647474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab553f44-677c-4506-b2ae-0161c69b3941 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:11.647745 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab553f44-677c-4506-b2ae-0161c69b3941 None admin] PATH_INFO: `/v3/users/glance` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:11.655146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab553f44-677c-4506-b2ae-0161c69b3941 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:11.656590 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab553f44-677c-4506-b2ae-0161c69b3941 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-ab553f44-677c-4506-b2ae-0161c69b3941 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:11.670464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:11.671525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:11.672686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:11.672686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:11.672686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:11.672686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:11.672686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:11.672686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:11.672686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:11.672686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:11.672686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 04:13:11.672686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:11.672686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 255/505] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:11.673236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f00de-e960-4e25-a011-033c3f4428f1 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:11.673236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f00de-e960-4e25-a011-033c3f4428f1 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:11.673350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f00de-e960-4e25-a011-033c3f4428f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:11.676913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a6fae-c8fb-4cca-b2e9-063844fd1fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:11.677581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a6fae-c8fb-4cca-b2e9-063844fd1fe2 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:11.677865 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a6fae-c8fb-4cca-b2e9-063844fd1fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:11.678140 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a6fae-c8fb-4cca-b2e9-063844fd1fe2 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:11.678885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a6fae-c8fb-4cca-b2e9-063844fd1fe2 None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:11.679196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a6fae-c8fb-4cca-b2e9-063844fd1fe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:11.680584 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a6fae-c8fb-4cca-b2e9-063844fd1fe2 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:11.689308 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 251/506] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:11.690115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a6fae-c8fb-4cca-b2e9-063844fd1fe2 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:11.691782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-062f00de-e960-4e25-a011-033c3f4428f1 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:11.693010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-062f00de-e960-4e25-a011-033c3f4428f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:11.761509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 256/507] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:11.762344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f00de-e960-4e25-a011-033c3f4428f1 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:11.790392 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a6fae-c8fb-4cca-b2e9-063844fd1fe2 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:11.822327 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50634f-907b-48b7-8e9e-7cb16f8e9cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:11.822989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50634f-907b-48b7-8e9e-7cb16f8e9cd0 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:11.823270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50634f-907b-48b7-8e9e-7cb16f8e9cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:11.823548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50634f-907b-48b7-8e9e-7cb16f8e9cd0 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:11.824296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50634f-907b-48b7-8e9e-7cb16f8e9cd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:11.824604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50634f-907b-48b7-8e9e-7cb16f8e9cd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:11.826008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50634f-907b-48b7-8e9e-7cb16f8e9cd0 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:11.837464 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 252/508] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:11.838245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50634f-907b-48b7-8e9e-7cb16f8e9cd0 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:11.854023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f00de-e960-4e25-a011-033c3f4428f1 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:11.885990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0a36d-d36b-45bd-a82a-32503ba32d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:11.886766 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0a36d-d36b-45bd-a82a-32503ba32d8c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:11.887046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0a36d-d36b-45bd-a82a-32503ba32d8c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:11.887317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0a36d-d36b-45bd-a82a-32503ba32d8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:11.897005 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0a36d-d36b-45bd-a82a-32503ba32d8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:11.898439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0a36d-d36b-45bd-a82a-32503ba32d8c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-bbc0a36d-d36b-45bd-a82a-32503ba32d8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:11.915194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:11.916098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:11.916923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:11.916923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:11.916923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:11.916923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:11.916923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 257/509] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:11.917516 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0a36d-d36b-45bd-a82a-32503ba32d8c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:11.938857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50634f-907b-48b7-8e9e-7cb16f8e9cd0 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:11.970797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2135c1-e0b7-4c5a-bb2d-4c894fa6ee6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:11.971461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2135c1-e0b7-4c5a-bb2d-4c894fa6ee6f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:11.971745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2135c1-e0b7-4c5a-bb2d-4c894fa6ee6f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:11.972054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2135c1-e0b7-4c5a-bb2d-4c894fa6ee6f None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:11.972876 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2135c1-e0b7-4c5a-bb2d-4c894fa6ee6f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:11.973184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2135c1-e0b7-4c5a-bb2d-4c894fa6ee6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:11.974600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2135c1-e0b7-4c5a-bb2d-4c894fa6ee6f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:11.991248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 253/510] 10.222.0.44 () {56 vars in 1137 bytes} [Mon May 9 04:13:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:11.991859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2135c1-e0b7-4c5a-bb2d-4c894fa6ee6f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:12.009791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0a36d-d36b-45bd-a82a-32503ba32d8c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:12.041480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2a17d-5f3e-408f-9386-2b0273409ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:12.042272 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2a17d-5f3e-408f-9386-2b0273409ada None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:12.042546 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2a17d-5f3e-408f-9386-2b0273409ada None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:12.042818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2a17d-5f3e-408f-9386-2b0273409ada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:12.052807 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2a17d-5f3e-408f-9386-2b0273409ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:12.054245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2a17d-5f3e-408f-9386-2b0273409ada None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-cae2a17d-5f3e-408f-9386-2b0273409ada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:12.070725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:12.071647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:12.072544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:12.072544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:12.072544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:12.072544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:12.072544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 258/511] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:12.072797 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2a17d-5f3e-408f-9386-2b0273409ada None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:12.084600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2135c1-e0b7-4c5a-bb2d-4c894fa6ee6f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:12.116637 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d652b-eb90-4880-abdf-2e8440212259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:12.117418 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d652b-eb90-4880-abdf-2e8440212259 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:12.117698 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d652b-eb90-4880-abdf-2e8440212259 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:12.117966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d652b-eb90-4880-abdf-2e8440212259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:12.127769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d652b-eb90-4880-abdf-2e8440212259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:12.129226 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d652b-eb90-4880-abdf-2e8440212259 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-206d652b-eb90-4880-abdf-2e8440212259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:12.145594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:12.146403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:12.147240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:12.147240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:12.147240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:12.147240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:12.147240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 254/512] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:12.147511 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d652b-eb90-4880-abdf-2e8440212259 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:12.174298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2a17d-5f3e-408f-9386-2b0273409ada None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:12.205509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523670d1-8445-4497-b4c9-4252a971fd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:12.206294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523670d1-8445-4497-b4c9-4252a971fd60 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:12.206583 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523670d1-8445-4497-b4c9-4252a971fd60 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:12.206858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523670d1-8445-4497-b4c9-4252a971fd60 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:12.213913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523670d1-8445-4497-b4c9-4252a971fd60 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:12.215349 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523670d1-8445-4497-b4c9-4252a971fd60 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-523670d1-8445-4497-b4c9-4252a971fd60 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:12.229859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:12.230847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:12.231751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:12.231751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:12.231751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:12.231751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:12.231751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:12.231751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:12.231751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:12.231751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:12.231751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 04:13:12.231751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:12.231751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 259/513] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:12.233528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523670d1-8445-4497-b4c9-4252a971fd60 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:12.247304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d652b-eb90-4880-abdf-2e8440212259 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:12.278088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258d195-6676-45f4-bd20-c73fd577b112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:12.278676 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258d195-6676-45f4-bd20-c73fd577b112 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:12.278958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258d195-6676-45f4-bd20-c73fd577b112 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:12.279786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258d195-6676-45f4-bd20-c73fd577b112 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:12.280644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258d195-6676-45f4-bd20-c73fd577b112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:12.280947 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258d195-6676-45f4-bd20-c73fd577b112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:12.282296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258d195-6676-45f4-bd20-c73fd577b112 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:12.293160 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 255/514] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:12.293942 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258d195-6676-45f4-bd20-c73fd577b112 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:12.325625 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523670d1-8445-4497-b4c9-4252a971fd60 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:12.358090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6490d24-940e-4f8f-ae69-904e2e128026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:12.358722 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6490d24-940e-4f8f-ae69-904e2e128026 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:12.359000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6490d24-940e-4f8f-ae69-904e2e128026 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:12.359273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6490d24-940e-4f8f-ae69-904e2e128026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:12.386482 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258d195-6676-45f4-bd20-c73fd577b112 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:12.418254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f4d62-c053-4559-9228-1dabdf63473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:12.418905 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f4d62-c053-4559-9228-1dabdf63473f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:12.419188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f4d62-c053-4559-9228-1dabdf63473f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:12.419461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f4d62-c053-4559-9228-1dabdf63473f None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:12.420441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f4d62-c053-4559-9228-1dabdf63473f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:12.420704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f4d62-c053-4559-9228-1dabdf63473f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:12.421218 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6490d24-940e-4f8f-ae69-904e2e128026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:12.422055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f4d62-c053-4559-9228-1dabdf63473f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:12.422653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6490d24-940e-4f8f-ae69-904e2e128026 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:12.432608 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 256/515] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:12.433390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f4d62-c053-4559-9228-1dabdf63473f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:12.512817 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 260/516] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:13:12.515500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6490d24-940e-4f8f-ae69-904e2e128026 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:12.524456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f4d62-c053-4559-9228-1dabdf63473f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:12.555856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40fb79-d612-4134-91d8-e9739192d404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:12.556469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40fb79-d612-4134-91d8-e9739192d404 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:12.556748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40fb79-d612-4134-91d8-e9739192d404 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:12.557025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40fb79-d612-4134-91d8-e9739192d404 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:12.557731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40fb79-d612-4134-91d8-e9739192d404 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:12.558030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40fb79-d612-4134-91d8-e9739192d404 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:12.559394 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40fb79-d612-4134-91d8-e9739192d404 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:12.570670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 257/517] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:12.571453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40fb79-d612-4134-91d8-e9739192d404 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:12.606896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6490d24-940e-4f8f-ae69-904e2e128026 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:12.639075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e52e6-53c0-47a9-adfa-151198f6fab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:12.639732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e52e6-53c0-47a9-adfa-151198f6fab5 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:12.640041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e52e6-53c0-47a9-adfa-151198f6fab5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:12.640317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e52e6-53c0-47a9-adfa-151198f6fab5 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:12.641145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e52e6-53c0-47a9-adfa-151198f6fab5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:12.641449 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e52e6-53c0-47a9-adfa-151198f6fab5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:12.642868 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e52e6-53c0-47a9-adfa-151198f6fab5 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:12.665091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40fb79-d612-4134-91d8-e9739192d404 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:12.677612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 261/518] 10.222.0.44 () {56 vars in 1137 bytes} [Mon May 9 04:13:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:12.678442 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e52e6-53c0-47a9-adfa-151198f6fab5 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:12.702353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208205e-284e-4ae7-a934-cbb6042dcd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:12.703141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208205e-284e-4ae7-a934-cbb6042dcd0c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:12.703668 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208205e-284e-4ae7-a934-cbb6042dcd0c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:12.703977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208205e-284e-4ae7-a934-cbb6042dcd0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:12.710291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208205e-284e-4ae7-a934-cbb6042dcd0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:12.711715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208205e-284e-4ae7-a934-cbb6042dcd0c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-c208205e-284e-4ae7-a934-cbb6042dcd0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:12.725886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:12.726831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:12.727681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:12.727681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:12.727681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:12.727681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:12.727681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:12.727681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:12.727681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:12.727681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:12.727681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 258/519] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:12.728184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208205e-284e-4ae7-a934-cbb6042dcd0c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:12.775142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e52e6-53c0-47a9-adfa-151198f6fab5 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:12.806932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc598759-14a5-4b27-b274-fb72e53ffb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:12.807629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc598759-14a5-4b27-b274-fb72e53ffb02 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:12.807931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc598759-14a5-4b27-b274-fb72e53ffb02 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:12.808201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc598759-14a5-4b27-b274-fb72e53ffb02 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:12.814859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc598759-14a5-4b27-b274-fb72e53ffb02 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:12.816325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc598759-14a5-4b27-b274-fb72e53ffb02 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:12.825995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208205e-284e-4ae7-a934-cbb6042dcd0c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-cc598759-14a5-4b27-b274-fb72e53ffb02 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:12.830346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:12.831213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:12.832190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:12.832190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:12.832190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:12.832190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:12.832190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:12.832190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:12.832190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:12.832190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:12.832190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 04:13:12.832190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:12.832190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 262/520] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:12.832753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc598759-14a5-4b27-b274-fb72e53ffb02 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:12.862626 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6e0b8-8135-4408-9420-bc22575ce0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:12.863421 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6e0b8-8135-4408-9420-bc22575ce0f9 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:12.863704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6e0b8-8135-4408-9420-bc22575ce0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:12.864010 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6e0b8-8135-4408-9420-bc22575ce0f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:12.874121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6e0b8-8135-4408-9420-bc22575ce0f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:12.875542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6e0b8-8135-4408-9420-bc22575ce0f9 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-fcb6e0b8-8135-4408-9420-bc22575ce0f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:12.891761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:12.892649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:12.893472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:12.893472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:12.893472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:12.893472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:12.893472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 259/521] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:12.893749 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6e0b8-8135-4408-9420-bc22575ce0f9 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:12.932526 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc598759-14a5-4b27-b274-fb72e53ffb02 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:12.963804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf91608-77f9-4641-99d3-adf91185be14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:12.964495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf91608-77f9-4641-99d3-adf91185be14 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:12.964775 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf91608-77f9-4641-99d3-adf91185be14 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:12.965050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf91608-77f9-4641-99d3-adf91185be14 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:12.965746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf91608-77f9-4641-99d3-adf91185be14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:12.967847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf91608-77f9-4641-99d3-adf91185be14 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:12.986798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 263/522] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:13:12 2022] POST /identity/v3/users => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:13:12.987426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf91608-77f9-4641-99d3-adf91185be14 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:12.988157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6e0b8-8135-4408-9420-bc22575ce0f9 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:13.028881 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c8821-9359-42db-86b0-06dfa9b843cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:13.030211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c8821-9359-42db-86b0-06dfa9b843cf None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:13.030500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c8821-9359-42db-86b0-06dfa9b843cf None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:13.030928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c8821-9359-42db-86b0-06dfa9b843cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:13.097129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf91608-77f9-4641-99d3-adf91185be14 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:13.111439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c8821-9359-42db-86b0-06dfa9b843cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:13.113337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c8821-9359-42db-86b0-06dfa9b843cf None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:13.133400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0143f-a3ed-4fae-805c-fdfa128d0ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:13.134405 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0143f-a3ed-4fae-805c-fdfa128d0ae4 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:13.134685 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0143f-a3ed-4fae-805c-fdfa128d0ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:13.134964 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0143f-a3ed-4fae-805c-fdfa128d0ae4 None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:13.135767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0143f-a3ed-4fae-805c-fdfa128d0ae4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:13.136119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0143f-a3ed-4fae-805c-fdfa128d0ae4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:13.137585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0143f-a3ed-4fae-805c-fdfa128d0ae4 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:13.158577 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 264/523] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:13.159844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0143f-a3ed-4fae-805c-fdfa128d0ae4 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:13.233811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 260/524] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:13:13.234919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c8821-9359-42db-86b0-06dfa9b843cf None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:13.269312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0143f-a3ed-4fae-805c-fdfa128d0ae4 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:13.301780 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7787a-1e4d-4b13-a9be-9d5c76dd532d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:13.302473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7787a-1e4d-4b13-a9be-9d5c76dd532d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:13.302759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7787a-1e4d-4b13-a9be-9d5c76dd532d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:13.303033 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7787a-1e4d-4b13-a9be-9d5c76dd532d None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:13.303867 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7787a-1e4d-4b13-a9be-9d5c76dd532d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:13.304199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7787a-1e4d-4b13-a9be-9d5c76dd532d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:13.305632 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7787a-1e4d-4b13-a9be-9d5c76dd532d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:13.327126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 265/525] 10.222.0.44 () {56 vars in 1139 bytes} [Mon May 9 04:13:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:13.327126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7787a-1e4d-4b13-a9be-9d5c76dd532d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:13.336222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c8821-9359-42db-86b0-06dfa9b843cf None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:13.369073 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4bc75-e7ed-4645-8405-c0ba0cd0967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:13.369664 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4bc75-e7ed-4645-8405-c0ba0cd0967f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:13.369946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4bc75-e7ed-4645-8405-c0ba0cd0967f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:13.370219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4bc75-e7ed-4645-8405-c0ba0cd0967f None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:13.370932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4bc75-e7ed-4645-8405-c0ba0cd0967f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:13.371238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4bc75-e7ed-4645-8405-c0ba0cd0967f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:13.372662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4bc75-e7ed-4645-8405-c0ba0cd0967f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:13.384102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 261/526] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:13.384908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4bc75-e7ed-4645-8405-c0ba0cd0967f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:13.429708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7787a-1e4d-4b13-a9be-9d5c76dd532d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:13.464111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac826c-bd96-4ed5-bc05-309eef999d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:13.464913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac826c-bd96-4ed5-bc05-309eef999d12 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:13.465262 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac826c-bd96-4ed5-bc05-309eef999d12 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:13.465606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac826c-bd96-4ed5-bc05-309eef999d12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:13.466541 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac826c-bd96-4ed5-bc05-309eef999d12 None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=faf6107735424ff7901614953e7df403) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:13.467438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac826c-bd96-4ed5-bc05-309eef999d12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:13.469171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac826c-bd96-4ed5-bc05-309eef999d12 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:13.474620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 266/527] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:13 2022] GET /identity/v3/role_assignments?user.id=faf6107735424ff7901614953e7df403&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:13.475404 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac826c-bd96-4ed5-bc05-309eef999d12 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:13.493987 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4bc75-e7ed-4645-8405-c0ba0cd0967f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:13.543058 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1958f-d676-4885-bb9f-db8c61efcd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:13.543917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1958f-d676-4885-bb9f-db8c61efcd77 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:13.544263 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1958f-d676-4885-bb9f-db8c61efcd77 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:13.544482 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1958f-d676-4885-bb9f-db8c61efcd77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:13.557774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1958f-d676-4885-bb9f-db8c61efcd77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:13.559416 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1958f-d676-4885-bb9f-db8c61efcd77 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-c5d1958f-d676-4885-bb9f-db8c61efcd77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:13.580677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:13.581764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:13.582796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:13.582796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:13.582796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:13.582796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:13.583544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 262/528] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:13.584578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1958f-d676-4885-bb9f-db8c61efcd77 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:13.594104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac826c-bd96-4ed5-bc05-309eef999d12 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:13.631365 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed8d5c-fa01-4dbd-adfd-1b2bc852ad1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:13.631970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed8d5c-fa01-4dbd-adfd-1b2bc852ad1d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:13.632311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed8d5c-fa01-4dbd-adfd-1b2bc852ad1d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:13.632599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed8d5c-fa01-4dbd-adfd-1b2bc852ad1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:13.695739 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1958f-d676-4885-bb9f-db8c61efcd77 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:13.715540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed8d5c-fa01-4dbd-adfd-1b2bc852ad1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:13.717687 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed8d5c-fa01-4dbd-adfd-1b2bc852ad1d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:13.732366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783e03a-6902-49cf-9f0a-d26f1412ee3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:13.734305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783e03a-6902-49cf-9f0a-d26f1412ee3f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:13.734817 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783e03a-6902-49cf-9f0a-d26f1412ee3f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:13.735310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783e03a-6902-49cf-9f0a-d26f1412ee3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:13.744511 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783e03a-6902-49cf-9f0a-d26f1412ee3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:13.746246 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783e03a-6902-49cf-9f0a-d26f1412ee3f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-7783e03a-6902-49cf-9f0a-d26f1412ee3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:13.762893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:13.764307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:13.765165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:13.765165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:13.765165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:13.765165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:13.765165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:13.765165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:13.765165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:13.765165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:13.767287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 263/529] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:13.768322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783e03a-6902-49cf-9f0a-d26f1412ee3f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:13.863979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 267/530] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:13:13.865206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed8d5c-fa01-4dbd-adfd-1b2bc852ad1d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:13.880176 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783e03a-6902-49cf-9f0a-d26f1412ee3f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:13.928060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b692ed0-8443-4a58-ba17-d5e7ae666ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:13.928941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b692ed0-8443-4a58-ba17-d5e7ae666ff4 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:13.929486 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b692ed0-8443-4a58-ba17-d5e7ae666ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:13.929770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b692ed0-8443-4a58-ba17-d5e7ae666ff4 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:13.930499 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b692ed0-8443-4a58-ba17-d5e7ae666ff4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:13.930802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b692ed0-8443-4a58-ba17-d5e7ae666ff4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:13.932211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b692ed0-8443-4a58-ba17-d5e7ae666ff4 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:13.947979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 264/531] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:13.949136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b692ed0-8443-4a58-ba17-d5e7ae666ff4 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:13.980120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed8d5c-fa01-4dbd-adfd-1b2bc852ad1d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:14.027828 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eac2b6-b231-407d-ab95-603cf6a5a090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:14.028540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eac2b6-b231-407d-ab95-603cf6a5a090 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:14.029503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eac2b6-b231-407d-ab95-603cf6a5a090 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:14.029503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eac2b6-b231-407d-ab95-603cf6a5a090 None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:14.030258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eac2b6-b231-407d-ab95-603cf6a5a090 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:14.030573 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eac2b6-b231-407d-ab95-603cf6a5a090 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:14.032037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eac2b6-b231-407d-ab95-603cf6a5a090 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:14.049445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 268/532] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:14.050328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eac2b6-b231-407d-ab95-603cf6a5a090 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:14.061522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b692ed0-8443-4a58-ba17-d5e7ae666ff4 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:14.095421 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa702d-9d3c-43e5-aa1a-1eefaf08afc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:14.096586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa702d-9d3c-43e5-aa1a-1eefaf08afc0 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:14.097104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa702d-9d3c-43e5-aa1a-1eefaf08afc0 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:14.097592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa702d-9d3c-43e5-aa1a-1eefaf08afc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:14.109014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa702d-9d3c-43e5-aa1a-1eefaf08afc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:14.110713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa702d-9d3c-43e5-aa1a-1eefaf08afc0 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-20fa702d-9d3c-43e5-aa1a-1eefaf08afc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:14.128462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:14.129369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:14.130200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:14.130200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:14.130200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:14.130200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:14.136605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 265/533] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:14.137687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa702d-9d3c-43e5-aa1a-1eefaf08afc0 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:14.150534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eac2b6-b231-407d-ab95-603cf6a5a090 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:14.184362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee2a84-ac60-4521-bdd7-22b53133c6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:14.185092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee2a84-ac60-4521-bdd7-22b53133c6dc None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:14.185408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee2a84-ac60-4521-bdd7-22b53133c6dc None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:14.185724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee2a84-ac60-4521-bdd7-22b53133c6dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:14.193178 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee2a84-ac60-4521-bdd7-22b53133c6dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:14.194687 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee2a84-ac60-4521-bdd7-22b53133c6dc None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-dfee2a84-ac60-4521-bdd7-22b53133c6dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:14.211674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:14.212599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:14.213432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:14.213432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:14.213432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:14.213432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:14.213432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:14.213432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:14.213432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:14.213432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:14.218402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 269/534] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:14.219392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee2a84-ac60-4521-bdd7-22b53133c6dc None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:14.248623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa702d-9d3c-43e5-aa1a-1eefaf08afc0 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:14.283705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8daac53-725b-465e-a791-58ca18169632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:14.284756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8daac53-725b-465e-a791-58ca18169632 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:14.285281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8daac53-725b-465e-a791-58ca18169632 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:14.285756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8daac53-725b-465e-a791-58ca18169632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:14.286780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8daac53-725b-465e-a791-58ca18169632 None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=295bb46627e043989e6dba2805607c99) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:14.287292 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8daac53-725b-465e-a791-58ca18169632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:14.289014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8daac53-725b-465e-a791-58ca18169632 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:14.295402 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 266/535] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:14 2022] GET /identity/v3/role_assignments?user.id=295bb46627e043989e6dba2805607c99&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:14.296527 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8daac53-725b-465e-a791-58ca18169632 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:14.338360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee2a84-ac60-4521-bdd7-22b53133c6dc None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:14.402287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4f1bf-6965-4e06-a609-a7640a7ee652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:14.402973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4f1bf-6965-4e06-a609-a7640a7ee652 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:14.403257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4f1bf-6965-4e06-a609-a7640a7ee652 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:14.403536 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4f1bf-6965-4e06-a609-a7640a7ee652 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:14.404341 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4f1bf-6965-4e06-a609-a7640a7ee652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:14.404693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4f1bf-6965-4e06-a609-a7640a7ee652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:14.405277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8daac53-725b-465e-a791-58ca18169632 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:14.406209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4f1bf-6965-4e06-a609-a7640a7ee652 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:14.427029 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 270/536] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:14.432550 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4f1bf-6965-4e06-a609-a7640a7ee652 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:14.447534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca42c1-a1ad-46b3-a388-ecd1318451a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:14.448158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca42c1-a1ad-46b3-a388-ecd1318451a6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:14.448439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca42c1-a1ad-46b3-a388-ecd1318451a6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:14.448711 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca42c1-a1ad-46b3-a388-ecd1318451a6 None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:14.449690 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca42c1-a1ad-46b3-a388-ecd1318451a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:14.449998 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca42c1-a1ad-46b3-a388-ecd1318451a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:14.452059 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca42c1-a1ad-46b3-a388-ecd1318451a6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:14.471125 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 267/537] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:14.486102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca42c1-a1ad-46b3-a388-ecd1318451a6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:14.542816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4f1bf-6965-4e06-a609-a7640a7ee652 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:14.578779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7b550-e9ab-4766-963a-6aa10ebd9002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:14.579592 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7b550-e9ab-4766-963a-6aa10ebd9002 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:14.579878 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7b550-e9ab-4766-963a-6aa10ebd9002 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:14.580181 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7b550-e9ab-4766-963a-6aa10ebd9002 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:14.587199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7b550-e9ab-4766-963a-6aa10ebd9002 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:14.588672 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7b550-e9ab-4766-963a-6aa10ebd9002 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-1eb7b550-e9ab-4766-963a-6aa10ebd9002 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:14.603537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:14.604490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:14.605327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:14.605327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:14.605327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:14.605327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:14.605327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:14.605327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:14.605327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:14.605327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:14.605327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 271/538] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:14.614598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7b550-e9ab-4766-963a-6aa10ebd9002 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:14.631123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca42c1-a1ad-46b3-a388-ecd1318451a6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:14.669518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310c461-a484-4e9c-b5bb-d11d19f02464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:14.670709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310c461-a484-4e9c-b5bb-d11d19f02464 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:14.671837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310c461-a484-4e9c-b5bb-d11d19f02464 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:14.671837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310c461-a484-4e9c-b5bb-d11d19f02464 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:14.673027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310c461-a484-4e9c-b5bb-d11d19f02464 None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=72caf90f411f495bb22aa85df4d25eb3) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:14.673341 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310c461-a484-4e9c-b5bb-d11d19f02464 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:14.674774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c310c461-a484-4e9c-b5bb-d11d19f02464 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:14.681780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 268/539] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:14 2022] GET /identity/v3/role_assignments?user.id=72caf90f411f495bb22aa85df4d25eb3&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:14.743309 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7b550-e9ab-4766-963a-6aa10ebd9002 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:14.782282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f433ec4-2b10-4ae8-aa4a-2afa635698eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:14.782948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f433ec4-2b10-4ae8-aa4a-2afa635698eb None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:14.783228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f433ec4-2b10-4ae8-aa4a-2afa635698eb None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:14.783503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f433ec4-2b10-4ae8-aa4a-2afa635698eb None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:14.784318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f433ec4-2b10-4ae8-aa4a-2afa635698eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:14.784623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f433ec4-2b10-4ae8-aa4a-2afa635698eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:14.786093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f433ec4-2b10-4ae8-aa4a-2afa635698eb None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:14.799432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 272/540] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:14.805364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310c461-a484-4e9c-b5bb-d11d19f02464 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:14.925247 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310c461-a484-4e9c-b5bb-d11d19f02464 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:14.975260 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827d060-ea29-44ad-b3f3-9e1e1795616b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:14.975963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827d060-ea29-44ad-b3f3-9e1e1795616b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:14.976248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827d060-ea29-44ad-b3f3-9e1e1795616b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:14.976525 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827d060-ea29-44ad-b3f3-9e1e1795616b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:14.977352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827d060-ea29-44ad-b3f3-9e1e1795616b None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=0d852975a2e84925a45775a16cc52f32) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:14.977658 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827d060-ea29-44ad-b3f3-9e1e1795616b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:14.979081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827d060-ea29-44ad-b3f3-9e1e1795616b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:14.985579 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 269/541] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:14 2022] GET /identity/v3/role_assignments?user.id=0d852975a2e84925a45775a16cc52f32&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:15.439135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5754f11-a53e-4e34-b2fb-6c0cf9c7b61e None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:15.439513 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5754f11-a53e-4e34-b2fb-6c0cf9c7b61e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:15.439681 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5754f11-a53e-4e34-b2fb-6c0cf9c7b61e None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:15.441067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 273/542] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:15.449561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993796fa-3a6e-4935-b1e1-2699be69bfad None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:15.451532 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993796fa-3a6e-4935-b1e1-2699be69bfad None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:15.451532 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993796fa-3a6e-4935-b1e1-2699be69bfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:15.467685 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-993796fa-3a6e-4935-b1e1-2699be69bfad None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:15.469216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-993796fa-3a6e-4935-b1e1-2699be69bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:15.553546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 270/543] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:15.598855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5754f11-a53e-4e34-b2fb-6c0cf9c7b61e None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:15.724618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5754f11-a53e-4e34-b2fb-6c0cf9c7b61e None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:15.775077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caa8d8-b3b5-422a-b4de-2143fbb152fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:15.775967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caa8d8-b3b5-422a-b4de-2143fbb152fa None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:15.776252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caa8d8-b3b5-422a-b4de-2143fbb152fa None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:15.776523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caa8d8-b3b5-422a-b4de-2143fbb152fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:15.791528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81caa8d8-b3b5-422a-b4de-2143fbb152fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-81caa8d8-b3b5-422a-b4de-2143fbb152fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:15.800007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:15.800923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:15.801742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:15.801742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:15.801742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:15.801742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:15.801742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:15.801742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:15.801742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:15.801742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:15.801742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:15.801742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:15.801742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:15.801742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:15.808371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 274/544] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:15.817025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993796fa-3a6e-4935-b1e1-2699be69bfad None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:15.964786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993796fa-3a6e-4935-b1e1-2699be69bfad None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:16.008433 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea26859-405a-4e75-b0af-ebb3edb5543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:16.009024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea26859-405a-4e75-b0af-ebb3edb5543e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:16.009305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea26859-405a-4e75-b0af-ebb3edb5543e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:16.009580 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea26859-405a-4e75-b0af-ebb3edb5543e None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:16.010329 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea26859-405a-4e75-b0af-ebb3edb5543e None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:16.010629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea26859-405a-4e75-b0af-ebb3edb5543e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:16.012011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea26859-405a-4e75-b0af-ebb3edb5543e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:16.023398 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 271/545] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:16.028595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caa8d8-b3b5-422a-b4de-2143fbb152fa None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:16.086081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f99c71-b5a9-4d5e-9bf8-26f7b81b5514 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:16.086400 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f99c71-b5a9-4d5e-9bf8-26f7b81b5514 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:16.086649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f99c71-b5a9-4d5e-9bf8-26f7b81b5514 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:16.088292 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 272/546] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:13:16.094016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81cb75-7906-4c76-9f51-ee9be9516926 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:16.094306 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81cb75-7906-4c76-9f51-ee9be9516926 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:16.094594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81cb75-7906-4c76-9f51-ee9be9516926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:16.140964 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-6e81cb75-7906-4c76-9f51-ee9be9516926 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:16.142173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-6e81cb75-7906-4c76-9f51-ee9be9516926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:16.150619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caa8d8-b3b5-422a-b4de-2143fbb152fa None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:16.190709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63cfcd-78b9-4626-80c5-37368047f0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:16.191547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63cfcd-78b9-4626-80c5-37368047f0ff None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:16.191828 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63cfcd-78b9-4626-80c5-37368047f0ff None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:16.192127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63cfcd-78b9-4626-80c5-37368047f0ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:16.207790 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63cfcd-78b9-4626-80c5-37368047f0ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:16.209262 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63cfcd-78b9-4626-80c5-37368047f0ff None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-5a63cfcd-78b9-4626-80c5-37368047f0ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:16.234908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:16.235805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:16.236686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:16.236686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:16.236686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:16.236686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:16.236686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 275/547] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:16.241911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63cfcd-78b9-4626-80c5-37368047f0ff None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:16.244194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 273/548] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:13:16.344597 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81cb75-7906-4c76-9f51-ee9be9516926 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:16.363661 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63cfcd-78b9-4626-80c5-37368047f0ff None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:16.398858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681dd0e-ea71-4d6d-8028-8c5aa0341de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:16.399522 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681dd0e-ea71-4d6d-8028-8c5aa0341de5 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:16.399803 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681dd0e-ea71-4d6d-8028-8c5aa0341de5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:16.400114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681dd0e-ea71-4d6d-8028-8c5aa0341de5 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:16.400827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681dd0e-ea71-4d6d-8028-8c5aa0341de5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:16.401126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681dd0e-ea71-4d6d-8028-8c5aa0341de5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:16.402503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681dd0e-ea71-4d6d-8028-8c5aa0341de5 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:16.427518 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 276/549] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:16.433386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681dd0e-ea71-4d6d-8028-8c5aa0341de5 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:16.535071 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81cb75-7906-4c76-9f51-ee9be9516926 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:16.574307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd79bc-9f78-4ba9-99d5-d9f5b32d58e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:16.575467 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd79bc-9f78-4ba9-99d5-d9f5b32d58e5 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:16.576046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd79bc-9f78-4ba9-99d5-d9f5b32d58e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:16.577140 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd79bc-9f78-4ba9-99d5-d9f5b32d58e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:16.598278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd79bc-9f78-4ba9-99d5-d9f5b32d58e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-27bd79bc-9f78-4ba9-99d5-d9f5b32d58e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:16.607200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:16.608107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:16.608892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:16.608892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:16.608892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:16.608892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:16.608892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:16.608892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:16.608892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:16.608892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:16.608892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:16.608892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:16.608892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:16.608892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:16.611580 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 274/550] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:16.621240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd79bc-9f78-4ba9-99d5-d9f5b32d58e5 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:16.629504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681dd0e-ea71-4d6d-8028-8c5aa0341de5 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:16.672143 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ad480-c57f-4d51-869f-6fdaf6c74c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:16.672956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ad480-c57f-4d51-869f-6fdaf6c74c2f None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:16.673235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ad480-c57f-4d51-869f-6fdaf6c74c2f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:16.673503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ad480-c57f-4d51-869f-6fdaf6c74c2f None admin] PATH_INFO: `/v3/users/swift` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:16.689306 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ad480-c57f-4d51-869f-6fdaf6c74c2f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:16.690771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ad480-c57f-4d51-869f-6fdaf6c74c2f None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-c11ad480-c57f-4d51-869f-6fdaf6c74c2f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:16.713749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:16.714712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:16.715598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:16.715598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:16.715598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:16.715598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:16.715598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:16.715598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:16.715598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:16.715598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:16.715598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 04:13:16.715598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:16.724386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 277/551] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:13:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:16.731600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ad480-c57f-4d51-869f-6fdaf6c74c2f None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:16.748937 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd79bc-9f78-4ba9-99d5-d9f5b32d58e5 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:16.786578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb5722-3c35-45a3-b6f7-11e5c57cfcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:16.787779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb5722-3c35-45a3-b6f7-11e5c57cfcad None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:16.790151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb5722-3c35-45a3-b6f7-11e5c57cfcad None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:16.791312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb5722-3c35-45a3-b6f7-11e5c57cfcad None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:16.792327 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb5722-3c35-45a3-b6f7-11e5c57cfcad None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:16.795887 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb5722-3c35-45a3-b6f7-11e5c57cfcad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:16.797534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb5722-3c35-45a3-b6f7-11e5c57cfcad None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:16.806402 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 275/552] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:16.812322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb5722-3c35-45a3-b6f7-11e5c57cfcad None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:16.916674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ad480-c57f-4d51-869f-6fdaf6c74c2f None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:16.957322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb5722-3c35-45a3-b6f7-11e5c57cfcad None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:16.992336 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0d43-37fc-4fa9-828a-b76727e42452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:16.993531 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0d43-37fc-4fa9-828a-b76727e42452 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:16.994015 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0d43-37fc-4fa9-828a-b76727e42452 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:16.994482 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0d43-37fc-4fa9-828a-b76727e42452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.006271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e0d43-37fc-4fa9-828a-b76727e42452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:17.008148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e0d43-37fc-4fa9-828a-b76727e42452 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:17.012729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9fbe6-3965-4776-a6d4-931f9a598b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:17.014036 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9fbe6-3965-4776-a6d4-931f9a598b5c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.014505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9fbe6-3965-4776-a6d4-931f9a598b5c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.014895 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9fbe6-3965-4776-a6d4-931f9a598b5c None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.015709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9fbe6-3965-4776-a6d4-931f9a598b5c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:17.016060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9fbe6-3965-4776-a6d4-931f9a598b5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:17.017541 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9fbe6-3965-4776-a6d4-931f9a598b5c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-f45e0d43-37fc-4fa9-828a-b76727e42452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:17.031158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:17.032115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:17.032945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:17.032945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:17.032945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:17.032945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:17.035334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 276/553] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:17.041310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0d43-37fc-4fa9-828a-b76727e42452 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:17.058411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 278/554] 10.222.0.44 () {56 vars in 1135 bytes} [Mon May 9 04:13:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:17.065556 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9fbe6-3965-4776-a6d4-931f9a598b5c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:17.148945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0d43-37fc-4fa9-828a-b76727e42452 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:17.183948 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fb301-53c8-4aa1-87c5-421998b76ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:17.184950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fb301-53c8-4aa1-87c5-421998b76ec9 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.185999 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fb301-53c8-4aa1-87c5-421998b76ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.186524 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fb301-53c8-4aa1-87c5-421998b76ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.187480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fb301-53c8-4aa1-87c5-421998b76ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:17.188007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fb301-53c8-4aa1-87c5-421998b76ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:17.189586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fb301-53c8-4aa1-87c5-421998b76ec9 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:17.202757 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 277/555] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:17.208608 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fb301-53c8-4aa1-87c5-421998b76ec9 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:17.268089 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9fbe6-3965-4776-a6d4-931f9a598b5c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:17.304396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de756df5-b21f-4287-a03a-6bfa88611332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:17.305400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de756df5-b21f-4287-a03a-6bfa88611332 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.305902 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de756df5-b21f-4287-a03a-6bfa88611332 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.306376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de756df5-b21f-4287-a03a-6bfa88611332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.335846 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fb301-53c8-4aa1-87c5-421998b76ec9 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:17.371191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd2262-10f9-4bdd-9839-3a827a8736a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:17.372156 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd2262-10f9-4bdd-9839-3a827a8736a4 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.372462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd2262-10f9-4bdd-9839-3a827a8736a4 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.372702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd2262-10f9-4bdd-9839-3a827a8736a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.378791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de756df5-b21f-4287-a03a-6bfa88611332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:17.380155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd2262-10f9-4bdd-9839-3a827a8736a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:17.380808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de756df5-b21f-4287-a03a-6bfa88611332 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:17.381574 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd2262-10f9-4bdd-9839-3a827a8736a4 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-5bfd2262-10f9-4bdd-9839-3a827a8736a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:17.398540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:17.399487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:17.405732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:17.405732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:17.405732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:17.405732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:17.405732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:17.405732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:17.405732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:17.405732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:17.405732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 04:13:17.405732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:17.405732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 278/556] 10.222.0.44 () {56 vars in 1093 bytes} [Mon May 9 04:13:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:17.405732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3c2da-d666-4b74-b6f0-92011f6d93a6 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.405732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3c2da-d666-4b74-b6f0-92011f6d93a6 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.406419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3c2da-d666-4b74-b6f0-92011f6d93a6 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.407727 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 279/557] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:17.408582 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3c2da-d666-4b74-b6f0-92011f6d93a6 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:17.490212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 279/558] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 04:13:17.490212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dfb4d-b274-4dcc-ba58-b16031e190f8 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.490212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dfb4d-b274-4dcc-ba58-b16031e190f8 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.490212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59dfb4d-b274-4dcc-ba58-b16031e190f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.508023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3c2da-d666-4b74-b6f0-92011f6d93a6 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:17.509975 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c59dfb4d-b274-4dcc-ba58-b16031e190f8 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:17.511137 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c59dfb4d-b274-4dcc-ba58-b16031e190f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:17.544043 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d89e73-8759-462f-80c1-f18c308f35e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:17.545288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d89e73-8759-462f-80c1-f18c308f35e6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.545584 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d89e73-8759-462f-80c1-f18c308f35e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.545857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d89e73-8759-462f-80c1-f18c308f35e6 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.546715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d89e73-8759-462f-80c1-f18c308f35e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:17.547046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d89e73-8759-462f-80c1-f18c308f35e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:17.548561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d89e73-8759-462f-80c1-f18c308f35e6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:17.571228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 280/559] 10.222.0.44 () {56 vars in 1133 bytes} [Mon May 9 04:13:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:17.571228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a1a1b-0bd4-4a42-b9b8-4743d120a2f0 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.571228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a1a1b-0bd4-4a42-b9b8-4743d120a2f0 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.571228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a1a1b-0bd4-4a42-b9b8-4743d120a2f0 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.571228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 281/560] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:17.571228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a1a1b-0bd4-4a42-b9b8-4743d120a2f0 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:17.592199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 280/561] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:17.592199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59dfb4d-b274-4dcc-ba58-b16031e190f8 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:17.684161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59dfb4d-b274-4dcc-ba58-b16031e190f8 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:17.688174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a1a1b-0bd4-4a42-b9b8-4743d120a2f0 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:17.717307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c63071-1ad1-4211-98f8-f56378b61d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:17.718107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c63071-1ad1-4211-98f8-f56378b61d41 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.718381 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c63071-1ad1-4211-98f8-f56378b61d41 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.718651 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c63071-1ad1-4211-98f8-f56378b61d41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.724255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314050a-861d-4314-9d84-1348d048a20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:17.725051 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314050a-861d-4314-9d84-1348d048a20c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.725330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314050a-861d-4314-9d84-1348d048a20c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.725602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314050a-861d-4314-9d84-1348d048a20c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.729052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c63071-1ad1-4211-98f8-f56378b61d41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:17.730562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c63071-1ad1-4211-98f8-f56378b61d41 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:17.735379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314050a-861d-4314-9d84-1348d048a20c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:17.736823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314050a-861d-4314-9d84-1348d048a20c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-e9c63071-1ad1-4211-98f8-f56378b61d41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:17.747992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:17.748974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:17.750019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:17.750019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:17.750019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:17.750019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:17.750273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 281/562] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:17.751354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc3586-366b-43f7-9d39-7186abac0fe4 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.751647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc3586-366b-43f7-9d39-7186abac0fe4 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.751947 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc3586-366b-43f7-9d39-7186abac0fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-b314050a-861d-4314-9d84-1348d048a20c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:17.754625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:17.755970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:17.756969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:17.756969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:17.756969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:17.756969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:17.756969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 282/563] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:17.756969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314050a-861d-4314-9d84-1348d048a20c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:17.773630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-31fc3586-366b-43f7-9d39-7186abac0fe4 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:17.774807 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-31fc3586-366b-43f7-9d39-7186abac0fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:17.837125 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 282/564] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:17.838435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe2f44-8cf7-498e-ab4f-95a47f785c28 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.838724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe2f44-8cf7-498e-ab4f-95a47f785c28 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.839002 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe2f44-8cf7-498e-ab4f-95a47f785c28 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.840614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 283/565] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:17.841252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe2f44-8cf7-498e-ab4f-95a47f785c28 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:17.856921 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314050a-861d-4314-9d84-1348d048a20c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:17.889603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f763b05-1e35-444a-b344-6d93eb4017da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:17.890399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f763b05-1e35-444a-b344-6d93eb4017da None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.890684 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f763b05-1e35-444a-b344-6d93eb4017da None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.890960 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f763b05-1e35-444a-b344-6d93eb4017da None admin] PATH_INFO: `/v3/roles/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.900495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f763b05-1e35-444a-b344-6d93eb4017da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-0f763b05-1e35-444a-b344-6d93eb4017da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:17.908844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:17.909913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:17.910733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:17.910733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:17.910733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:17.910733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:17.910733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:17.910733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:17.910733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:17.910733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:17.910733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:17.910733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:17.910733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:17.910733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:17.910733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 283/566] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:17.911783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f763b05-1e35-444a-b344-6d93eb4017da None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:17.936882 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe2f44-8cf7-498e-ab4f-95a47f785c28 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:17.968846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9781927-0642-4f9f-80b4-f3893d0d0564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:17.969505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9781927-0642-4f9f-80b4-f3893d0d0564 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.969785 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9781927-0642-4f9f-80b4-f3893d0d0564 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.970161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9781927-0642-4f9f-80b4-f3893d0d0564 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:17.970784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9781927-0642-4f9f-80b4-f3893d0d0564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:17.971063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9781927-0642-4f9f-80b4-f3893d0d0564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:17.972552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9781927-0642-4f9f-80b4-f3893d0d0564 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:17.983203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 284/567] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:17.984438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054a946-78e0-480d-bb02-0ae00ff98e22 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:17.984728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054a946-78e0-480d-bb02-0ae00ff98e22 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:17.985003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054a946-78e0-480d-bb02-0ae00ff98e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.001021 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-6054a946-78e0-480d-bb02-0ae00ff98e22 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:18.002133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6054a946-78e0-480d-bb02-0ae00ff98e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:18.007872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f763b05-1e35-444a-b344-6d93eb4017da None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:18.040009 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad370c-44ae-46cb-b696-6752fb06420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:18.040662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad370c-44ae-46cb-b696-6752fb06420c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:18.040941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad370c-44ae-46cb-b696-6752fb06420c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:18.041210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad370c-44ae-46cb-b696-6752fb06420c None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.041922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad370c-44ae-46cb-b696-6752fb06420c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:18.042226 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad370c-44ae-46cb-b696-6752fb06420c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:18.043566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad370c-44ae-46cb-b696-6752fb06420c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:18.056101 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 284/568] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:18.057352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad370c-44ae-46cb-b696-6752fb06420c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:18.068916 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 285/569] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:18.069747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054a946-78e0-480d-bb02-0ae00ff98e22 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:18.149678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad370c-44ae-46cb-b696-6752fb06420c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:18.166269 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054a946-78e0-480d-bb02-0ae00ff98e22 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:18.181550 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99785ec2-fb76-40a3-afd7-f8778e120065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:18.182344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99785ec2-fb76-40a3-afd7-f8778e120065 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:18.182622 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99785ec2-fb76-40a3-afd7-f8778e120065 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:18.182896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99785ec2-fb76-40a3-afd7-f8778e120065 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.192405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99785ec2-fb76-40a3-afd7-f8778e120065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:18.198167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e5702-03b4-4a29-b015-f2b3c9de6e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:18.198816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e5702-03b4-4a29-b015-f2b3c9de6e71 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:18.199093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e5702-03b4-4a29-b015-f2b3c9de6e71 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:18.199363 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e5702-03b4-4a29-b015-f2b3c9de6e71 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-99785ec2-fb76-40a3-afd7-f8778e120065 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:18.199965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:18.201116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:18.202001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:18.202001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:18.202001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:18.202001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:18.202001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:18.202001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:18.202001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:18.202001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:18.202001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:18.202001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:18.202001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:18.202001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:18.202613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e5702-03b4-4a29-b015-f2b3c9de6e71 None admin] RBAC: Adding query filter params (name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:18.202613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e5702-03b4-4a29-b015-f2b3c9de6e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:18.202734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 285/570] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:18.202799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e5702-03b4-4a29-b015-f2b3c9de6e71 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:18.202863 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99785ec2-fb76-40a3-afd7-f8778e120065 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:18.210044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 286/571] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:18.210822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e5702-03b4-4a29-b015-f2b3c9de6e71 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:18.296885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99785ec2-fb76-40a3-afd7-f8778e120065 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:18.302324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e5702-03b4-4a29-b015-f2b3c9de6e71 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:18.327419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb317ac-ff6f-49fc-adf8-db2285dc9920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:18.328237 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb317ac-ff6f-49fc-adf8-db2285dc9920 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:18.328781 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb317ac-ff6f-49fc-adf8-db2285dc9920 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:18.329054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb317ac-ff6f-49fc-adf8-db2285dc9920 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.333541 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9e7ac-6503-41c6-a418-45fd7f82727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:18.334320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9e7ac-6503-41c6-a418-45fd7f82727d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:18.334596 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9e7ac-6503-41c6-a418-45fd7f82727d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:18.334864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9e7ac-6503-41c6-a418-45fd7f82727d None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.335416 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb317ac-ff6f-49fc-adf8-db2285dc9920 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:18.336869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb317ac-ff6f-49fc-adf8-db2285dc9920 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:18.341207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e9e7ac-6503-41c6-a418-45fd7f82727d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:18.343279 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e9e7ac-6503-41c6-a418-45fd7f82727d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-1cb317ac-ff6f-49fc-adf8-db2285dc9920 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:18.350880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:18.352314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:18.353258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:18.353258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:18.353258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:18.353258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:18.353258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:18.353258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:18.353258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:18.353258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:18.353258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 286/572] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:18.353258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb317ac-ff6f-49fc-adf8-db2285dc9920 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-25e9e7ac-6503-41c6-a418-45fd7f82727d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:18.356879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:18.357872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:18.359006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:18.359006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:18.359006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:18.359006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:18.359006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:18.359006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:18.359006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:18.359006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:18.359006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 287/573] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:18.359664 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9e7ac-6503-41c6-a418-45fd7f82727d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:18.448883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb317ac-ff6f-49fc-adf8-db2285dc9920 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:18.449513 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9e7ac-6503-41c6-a418-45fd7f82727d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:18.481164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb697430-3773-42ac-b6ed-d91126507576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:18.481698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3d2ad-ef5b-425a-8960-68d419bf89ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:18.481945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb697430-3773-42ac-b6ed-d91126507576 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:18.482274 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb697430-3773-42ac-b6ed-d91126507576 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:18.482374 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3d2ad-ef5b-425a-8960-68d419bf89ea None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:18.482548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb697430-3773-42ac-b6ed-d91126507576 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.482639 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3d2ad-ef5b-425a-8960-68d419bf89ea None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:18.482905 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3d2ad-ef5b-425a-8960-68d419bf89ea None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.483644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3d2ad-ef5b-425a-8960-68d419bf89ea None admin] RBAC: Adding query filter params (name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:18.483971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3d2ad-ef5b-425a-8960-68d419bf89ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:18.485411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3d2ad-ef5b-425a-8960-68d419bf89ea None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:18.492124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb697430-3773-42ac-b6ed-d91126507576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:18.492943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 288/574] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:18.493718 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3d2ad-ef5b-425a-8960-68d419bf89ea None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-cb697430-3773-42ac-b6ed-d91126507576 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:18.499816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:18.500928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 287/575] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:18.501767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb697430-3773-42ac-b6ed-d91126507576 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:18.586127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3d2ad-ef5b-425a-8960-68d419bf89ea None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:18.596953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb697430-3773-42ac-b6ed-d91126507576 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:18.617109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bbaef-65f7-49b6-b9c5-ad2b08ab84d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:18.617889 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bbaef-65f7-49b6-b9c5-ad2b08ab84d1 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:18.618167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bbaef-65f7-49b6-b9c5-ad2b08ab84d1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:18.618436 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bbaef-65f7-49b6-b9c5-ad2b08ab84d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.627336 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae12a1-c0df-4027-93c3-1dfc902d83aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:18.628004 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae12a1-c0df-4027-93c3-1dfc902d83aa None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:18.628287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae12a1-c0df-4027-93c3-1dfc902d83aa None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:18.628425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bbaef-65f7-49b6-b9c5-ad2b08ab84d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:18.628563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae12a1-c0df-4027-93c3-1dfc902d83aa None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.629329 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae12a1-c0df-4027-93c3-1dfc902d83aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:18.629635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae12a1-c0df-4027-93c3-1dfc902d83aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:18.629832 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bbaef-65f7-49b6-b9c5-ad2b08ab84d1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:18.631024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae12a1-c0df-4027-93c3-1dfc902d83aa None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:18.642622 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 288/576] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:18.643397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae12a1-c0df-4027-93c3-1dfc902d83aa None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-e53bbaef-65f7-49b6-b9c5-ad2b08ab84d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:18.646694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:18.647547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:18.648478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:18.648478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:18.648478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:18.648478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:18.648478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 289/577] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:18.648796 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bbaef-65f7-49b6-b9c5-ad2b08ab84d1 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:18.739528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae12a1-c0df-4027-93c3-1dfc902d83aa None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:18.742472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bbaef-65f7-49b6-b9c5-ad2b08ab84d1 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:18.774907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1495fee-5fd2-4d43-8ca9-4bda01a0b1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:18.775255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaf34e-7375-4e09-a2d6-e0a72c1a8ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:18.775575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1495fee-5fd2-4d43-8ca9-4bda01a0b1c2 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:18.775862 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1495fee-5fd2-4d43-8ca9-4bda01a0b1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:18.776072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaf34e-7375-4e09-a2d6-e0a72c1a8ecb None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:18.776170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1495fee-5fd2-4d43-8ca9-4bda01a0b1c2 None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.776352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaf34e-7375-4e09-a2d6-e0a72c1a8ecb None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:18.776624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaf34e-7375-4e09-a2d6-e0a72c1a8ecb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.776935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1495fee-5fd2-4d43-8ca9-4bda01a0b1c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:18.777243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1495fee-5fd2-4d43-8ca9-4bda01a0b1c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:18.778667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1495fee-5fd2-4d43-8ca9-4bda01a0b1c2 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:18.786686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aaf34e-7375-4e09-a2d6-e0a72c1a8ecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:18.788140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aaf34e-7375-4e09-a2d6-e0a72c1a8ecb None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:18.791240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 289/578] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:18.792063 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1495fee-5fd2-4d43-8ca9-4bda01a0b1c2 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-75aaf34e-7375-4e09-a2d6-e0a72c1a8ecb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:18.804950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:18.806090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:18.807528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:18.807528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:18.807528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:18.807528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:18.808193 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 290/579] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:18.809124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaf34e-7375-4e09-a2d6-e0a72c1a8ecb None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:18.886820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1495fee-5fd2-4d43-8ca9-4bda01a0b1c2 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:18.905789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaf34e-7375-4e09-a2d6-e0a72c1a8ecb None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:18.918587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9f6df-0a32-41b7-9ca7-baf6bed9f1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:18.919246 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9f6df-0a32-41b7-9ca7-baf6bed9f1b7 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:18.919527 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9f6df-0a32-41b7-9ca7-baf6bed9f1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:18.920780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9f6df-0a32-41b7-9ca7-baf6bed9f1b7 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.921530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9f6df-0a32-41b7-9ca7-baf6bed9f1b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:18.921835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9f6df-0a32-41b7-9ca7-baf6bed9f1b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:18.923204 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9f6df-0a32-41b7-9ca7-baf6bed9f1b7 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:18.930692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 290/580] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:18.931465 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9f6df-0a32-41b7-9ca7-baf6bed9f1b7 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:18.938234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e1963-8610-4d82-bf73-3e3081a9f159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:18.939124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e1963-8610-4d82-bf73-3e3081a9f159 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:18.939407 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e1963-8610-4d82-bf73-3e3081a9f159 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:18.939688 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e1963-8610-4d82-bf73-3e3081a9f159 None admin] PATH_INFO: `/v3/projects/3322b00723974bd99e837d447bac22d1/users/faf6107735424ff7901614953e7df403/roles/c255c186230d4339b46d5e1f7f1f6cbc` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:18.954257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e1963-8610-4d82-bf73-3e3081a9f159 None admin] RBAC: Authorizing `identity:create_grant(project_id=3322b00723974bd99e837d447bac22d1, user_id=faf6107735424ff7901614953e7df403, role_id=c255c186230d4339b46d5e1f7f1f6cbc)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:18.955685 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e1963-8610-4d82-bf73-3e3081a9f159 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:18.968483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 291/581] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:13:18 2022] PUT /identity/v3/projects/3322b00723974bd99e837d447bac22d1/users/faf6107735424ff7901614953e7df403/roles/c255c186230d4339b46d5e1f7f1f6cbc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:13:18.969271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e1963-8610-4d82-bf73-3e3081a9f159 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:19.033541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9f6df-0a32-41b7-9ca7-baf6bed9f1b7 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:19.067836 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0771f0-bc13-4ccc-88ef-b9638347ba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:19.068547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0771f0-bc13-4ccc-88ef-b9638347ba99 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:19.068829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0771f0-bc13-4ccc-88ef-b9638347ba99 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:19.069107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0771f0-bc13-4ccc-88ef-b9638347ba99 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:19.069847 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0771f0-bc13-4ccc-88ef-b9638347ba99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:19.070153 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0771f0-bc13-4ccc-88ef-b9638347ba99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:19.071527 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0771f0-bc13-4ccc-88ef-b9638347ba99 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:19.079759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e1963-8610-4d82-bf73-3e3081a9f159 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:19.085101 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 291/582] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:19.086045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0771f0-bc13-4ccc-88ef-b9638347ba99 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:19.117204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0b936-5d04-4b2f-8908-b5915efdc5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:19.118213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0b936-5d04-4b2f-8908-b5915efdc5a1 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:19.118480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0b936-5d04-4b2f-8908-b5915efdc5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:19.118768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0b936-5d04-4b2f-8908-b5915efdc5a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:19.119686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0b936-5d04-4b2f-8908-b5915efdc5a1 None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=c46e7cc6407f44f08a659288e3ad0767) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:19.120036 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0b936-5d04-4b2f-8908-b5915efdc5a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:19.121508 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0b936-5d04-4b2f-8908-b5915efdc5a1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:19.133653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 292/583] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:18 2022] GET /identity/v3/role_assignments?user.id=c46e7cc6407f44f08a659288e3ad0767&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:19.133653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0b936-5d04-4b2f-8908-b5915efdc5a1 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:19.200630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0771f0-bc13-4ccc-88ef-b9638347ba99 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:19.250636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04f47e-60a9-484b-901b-4ae09e3ee329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:19.252152 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04f47e-60a9-484b-901b-4ae09e3ee329 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:19.252459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04f47e-60a9-484b-901b-4ae09e3ee329 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:19.252741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04f47e-60a9-484b-901b-4ae09e3ee329 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:19.253491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04f47e-60a9-484b-901b-4ae09e3ee329 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:19.253808 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04f47e-60a9-484b-901b-4ae09e3ee329 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:19.255206 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04f47e-60a9-484b-901b-4ae09e3ee329 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:19.261512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0b936-5d04-4b2f-8908-b5915efdc5a1 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:19.267824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 292/584] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:19.268713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04f47e-60a9-484b-901b-4ae09e3ee329 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:19.299706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d89b5c-b148-4f7a-86c4-4a475ddb992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:19.300985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d89b5c-b148-4f7a-86c4-4a475ddb992b None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:19.301500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d89b5c-b148-4f7a-86c4-4a475ddb992b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:19.301994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d89b5c-b148-4f7a-86c4-4a475ddb992b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:19.313970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d89b5c-b148-4f7a-86c4-4a475ddb992b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:19.315813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d89b5c-b148-4f7a-86c4-4a475ddb992b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-49d89b5c-b148-4f7a-86c4-4a475ddb992b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:19.335088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:19.336090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:19.336990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:19.336990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:19.336990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:19.336990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:19.339392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 293/585] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:19.340478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d89b5c-b148-4f7a-86c4-4a475ddb992b None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:19.374185 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04f47e-60a9-484b-901b-4ae09e3ee329 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:19.408461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5839e82-f7db-492d-81c6-4b01db80c299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:19.409267 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5839e82-f7db-492d-81c6-4b01db80c299 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:19.409549 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5839e82-f7db-492d-81c6-4b01db80c299 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:19.409824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5839e82-f7db-492d-81c6-4b01db80c299 None admin] PATH_INFO: `/v3/users/glance` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:19.417481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5839e82-f7db-492d-81c6-4b01db80c299 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:19.418904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5839e82-f7db-492d-81c6-4b01db80c299 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-c5839e82-f7db-492d-81c6-4b01db80c299 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:19.433030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:19.434145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:19.435115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:19.435115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:19.435115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:19.435115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:19.435115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:19.435115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:19.435115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:19.435115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:19.435115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 04:13:19.435115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:19.439107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 293/586] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:19.439977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5839e82-f7db-492d-81c6-4b01db80c299 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:19.453113 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d89b5c-b148-4f7a-86c4-4a475ddb992b None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:19.488289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f50e4-63ac-4a16-ac39-f7033a32f665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:19.489526 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f50e4-63ac-4a16-ac39-f7033a32f665 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:19.490060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f50e4-63ac-4a16-ac39-f7033a32f665 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:19.490711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f50e4-63ac-4a16-ac39-f7033a32f665 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:19.499172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f50e4-63ac-4a16-ac39-f7033a32f665 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:19.500975 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f50e4-63ac-4a16-ac39-f7033a32f665 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-9c5f50e4-63ac-4a16-ac39-f7033a32f665 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:19.517077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:19.518076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:19.519027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:19.519027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:19.519027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:19.519027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:19.519027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:19.519027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:19.519027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:19.519027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:19.519027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 04:13:19.519027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:19.521647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 294/587] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:19.522705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f50e4-63ac-4a16-ac39-f7033a32f665 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:19.541380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5839e82-f7db-492d-81c6-4b01db80c299 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:19.575205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffd5ea-df85-44c5-beed-370dbc7e22e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:19.576507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffd5ea-df85-44c5-beed-370dbc7e22e1 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:19.576831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffd5ea-df85-44c5-beed-370dbc7e22e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:19.577310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffd5ea-df85-44c5-beed-370dbc7e22e1 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:19.577811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffd5ea-df85-44c5-beed-370dbc7e22e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:19.578115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffd5ea-df85-44c5-beed-370dbc7e22e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:19.579726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffd5ea-df85-44c5-beed-370dbc7e22e1 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:19.591879 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 294/588] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:19.592789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffd5ea-df85-44c5-beed-370dbc7e22e1 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:19.629102 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f50e4-63ac-4a16-ac39-f7033a32f665 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:19.665269 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203cedd-906c-4ba4-a6ca-76d5e259880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:19.666862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203cedd-906c-4ba4-a6ca-76d5e259880e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:19.667387 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203cedd-906c-4ba4-a6ca-76d5e259880e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:19.667886 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203cedd-906c-4ba4-a6ca-76d5e259880e None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:19.668969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203cedd-906c-4ba4-a6ca-76d5e259880e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:19.669491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203cedd-906c-4ba4-a6ca-76d5e259880e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:19.671501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203cedd-906c-4ba4-a6ca-76d5e259880e None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:19.690559 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 295/589] 10.222.0.44 () {56 vars in 1137 bytes} [Mon May 9 04:13:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:19.691786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffd5ea-df85-44c5-beed-370dbc7e22e1 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:19.692201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203cedd-906c-4ba4-a6ca-76d5e259880e None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:19.731098 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81659e28-4d29-49cf-941e-f4966f4d6d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:19.731566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81659e28-4d29-49cf-941e-f4966f4d6d8f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:19.731848 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81659e28-4d29-49cf-941e-f4966f4d6d8f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:19.732223 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81659e28-4d29-49cf-941e-f4966f4d6d8f None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:19.733630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81659e28-4d29-49cf-941e-f4966f4d6d8f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:19.733867 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81659e28-4d29-49cf-941e-f4966f4d6d8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:19.735324 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81659e28-4d29-49cf-941e-f4966f4d6d8f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:19.764294 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 295/590] 10.222.0.44 () {56 vars in 1137 bytes} [Mon May 9 04:13:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:19.765215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81659e28-4d29-49cf-941e-f4966f4d6d8f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:19.805723 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203cedd-906c-4ba4-a6ca-76d5e259880e None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:19.840860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74359f41-f7e1-4471-8893-32611b9684a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:19.842579 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74359f41-f7e1-4471-8893-32611b9684a9 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:19.843253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74359f41-f7e1-4471-8893-32611b9684a9 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:19.843734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74359f41-f7e1-4471-8893-32611b9684a9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:19.851146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74359f41-f7e1-4471-8893-32611b9684a9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:19.852863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74359f41-f7e1-4471-8893-32611b9684a9 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:19.862879 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81659e28-4d29-49cf-941e-f4966f4d6d8f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-74359f41-f7e1-4471-8893-32611b9684a9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:19.868189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:19.869177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:19.870191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:19.870191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:19.870191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:19.870191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:19.870191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:19.870191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:19.870191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:19.870191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:19.870191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 04:13:19.870191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:19.873090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 296/591] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:19.874188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74359f41-f7e1-4471-8893-32611b9684a9 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:19.896949 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4932b8-7eb2-4347-b0b2-13c42a709db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:19.897649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4932b8-7eb2-4347-b0b2-13c42a709db6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:19.897934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4932b8-7eb2-4347-b0b2-13c42a709db6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:19.898211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4932b8-7eb2-4347-b0b2-13c42a709db6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:19.909285 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4932b8-7eb2-4347-b0b2-13c42a709db6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:19.910729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4932b8-7eb2-4347-b0b2-13c42a709db6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-1f4932b8-7eb2-4347-b0b2-13c42a709db6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:19.928578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:19.929544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:19.930445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:19.930445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:19.930445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:19.930445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:19.934379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 296/592] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:19.935116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4932b8-7eb2-4347-b0b2-13c42a709db6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:19.980194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74359f41-f7e1-4471-8893-32611b9684a9 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:20.015189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd049a-bd3d-4fbe-8d87-dbbb4f6c9a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:20.016368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd049a-bd3d-4fbe-8d87-dbbb4f6c9a80 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:20.016914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd049a-bd3d-4fbe-8d87-dbbb4f6c9a80 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:20.017404 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd049a-bd3d-4fbe-8d87-dbbb4f6c9a80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:20.027713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd049a-bd3d-4fbe-8d87-dbbb4f6c9a80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:20.029516 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd049a-bd3d-4fbe-8d87-dbbb4f6c9a80 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:20.033601 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4932b8-7eb2-4347-b0b2-13c42a709db6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-7ddd049a-bd3d-4fbe-8d87-dbbb4f6c9a80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:20.055997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:20.056967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:20.057934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:20.057934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:20.057934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:20.057934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:20.060223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 297/593] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:20.063775 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd049a-bd3d-4fbe-8d87-dbbb4f6c9a80 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:20.070099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01c2fe-8467-4db5-aced-7178fcb1feec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:20.070835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01c2fe-8467-4db5-aced-7178fcb1feec None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:20.071119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01c2fe-8467-4db5-aced-7178fcb1feec None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:20.071447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01c2fe-8467-4db5-aced-7178fcb1feec None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:20.072321 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01c2fe-8467-4db5-aced-7178fcb1feec None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:20.072630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01c2fe-8467-4db5-aced-7178fcb1feec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:20.074053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01c2fe-8467-4db5-aced-7178fcb1feec None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:20.093393 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 297/594] 10.222.0.44 () {56 vars in 1139 bytes} [Mon May 9 04:13:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:20.094238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01c2fe-8467-4db5-aced-7178fcb1feec None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:20.168806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd049a-bd3d-4fbe-8d87-dbbb4f6c9a80 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:20.194553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01c2fe-8467-4db5-aced-7178fcb1feec None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:20.210482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbe83f-92af-452c-a17e-be81ede40fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:20.211627 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbe83f-92af-452c-a17e-be81ede40fc3 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:20.212465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbe83f-92af-452c-a17e-be81ede40fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:20.212950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbe83f-92af-452c-a17e-be81ede40fc3 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:20.213899 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbe83f-92af-452c-a17e-be81ede40fc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:20.214417 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbe83f-92af-452c-a17e-be81ede40fc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:20.217321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbe83f-92af-452c-a17e-be81ede40fc3 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:20.234450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831b47c-43ed-4e97-b8a0-e7a6cdf7fbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:20.234795 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 298/595] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:20.235830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbe83f-92af-452c-a17e-be81ede40fc3 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:20.236736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831b47c-43ed-4e97-b8a0-e7a6cdf7fbc4 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:20.237237 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831b47c-43ed-4e97-b8a0-e7a6cdf7fbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:20.237820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831b47c-43ed-4e97-b8a0-e7a6cdf7fbc4 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:20.238760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831b47c-43ed-4e97-b8a0-e7a6cdf7fbc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:20.239283 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831b47c-43ed-4e97-b8a0-e7a6cdf7fbc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:20.241081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831b47c-43ed-4e97-b8a0-e7a6cdf7fbc4 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:20.253343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 298/596] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:20.254429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831b47c-43ed-4e97-b8a0-e7a6cdf7fbc4 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:20.355475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831b47c-43ed-4e97-b8a0-e7a6cdf7fbc4 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:20.358859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbe83f-92af-452c-a17e-be81ede40fc3 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:20.392799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced1dc4-87d1-42e7-a209-eef1090f0a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:20.394065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1dc4-87d1-42e7-a209-eef1090f0a78 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:20.394555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1dc4-87d1-42e7-a209-eef1090f0a78 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:20.395050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1dc4-87d1-42e7-a209-eef1090f0a78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:20.397111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38963c6-5c0f-41fe-aeba-a356266d18c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:20.398149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38963c6-5c0f-41fe-aeba-a356266d18c0 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:20.398740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38963c6-5c0f-41fe-aeba-a356266d18c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:20.399019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38963c6-5c0f-41fe-aeba-a356266d18c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:20.407587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38963c6-5c0f-41fe-aeba-a356266d18c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:20.408443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced1dc4-87d1-42e7-a209-eef1090f0a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:20.409048 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38963c6-5c0f-41fe-aeba-a356266d18c0 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:20.410380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced1dc4-87d1-42e7-a209-eef1090f0a78 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-f38963c6-5c0f-41fe-aeba-a356266d18c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:20.424195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:20.425114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:20.426000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:20.426000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:20.426000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:20.426000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:20.426000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:20.426000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:20.426000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:20.426000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:20.430150 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 299/597] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:20.430976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38963c6-5c0f-41fe-aeba-a356266d18c0 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-2ced1dc4-87d1-42e7-a209-eef1090f0a78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:20.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:20.432544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:20.433448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:20.433448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:20.433448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:20.433448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:20.436149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 299/598] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:20.437344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced1dc4-87d1-42e7-a209-eef1090f0a78 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:20.538814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced1dc4-87d1-42e7-a209-eef1090f0a78 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:20.544446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38963c6-5c0f-41fe-aeba-a356266d18c0 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:20.573521 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd010e0-33d1-437a-912c-a555005e9b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:20.574595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd010e0-33d1-437a-912c-a555005e9b9c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:20.575112 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd010e0-33d1-437a-912c-a555005e9b9c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:20.575689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd010e0-33d1-437a-912c-a555005e9b9c None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:20.577644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd010e0-33d1-437a-912c-a555005e9b9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:20.578296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd010e0-33d1-437a-912c-a555005e9b9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:20.579669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42478d85-da86-47f6-97b4-e910e0ce2973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:20.580398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd010e0-33d1-437a-912c-a555005e9b9c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:20.580686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42478d85-da86-47f6-97b4-e910e0ce2973 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:20.581065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42478d85-da86-47f6-97b4-e910e0ce2973 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:20.581434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42478d85-da86-47f6-97b4-e910e0ce2973 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:20.588467 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42478d85-da86-47f6-97b4-e910e0ce2973 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:20.589768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42478d85-da86-47f6-97b4-e910e0ce2973 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:20.594359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 300/599] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:20.595433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd010e0-33d1-437a-912c-a555005e9b9c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-42478d85-da86-47f6-97b4-e910e0ce2973 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:20.604081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:20.605201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:20.606330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:20.606330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:20.606330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:20.606330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:20.606330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:20.606330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:20.606330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:20.606330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:20.612506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 300/600] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:20.613439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42478d85-da86-47f6-97b4-e910e0ce2973 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:20.704157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd010e0-33d1-437a-912c-a555005e9b9c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:20.711913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42478d85-da86-47f6-97b4-e910e0ce2973 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:20.738805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1a893-1dc5-4660-81d3-ef534dc158a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:20.741952 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1a893-1dc5-4660-81d3-ef534dc158a6 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:20.741952 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1a893-1dc5-4660-81d3-ef534dc158a6 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:20.741952 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1a893-1dc5-4660-81d3-ef534dc158a6 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:20.741952 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1a893-1dc5-4660-81d3-ef534dc158a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:20.741952 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1a893-1dc5-4660-81d3-ef534dc158a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:20.743546 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1a893-1dc5-4660-81d3-ef534dc158a6 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:20.749538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a06f0-c1d7-43e9-9f4e-29dce47f0024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:20.750797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a06f0-c1d7-43e9-9f4e-29dce47f0024 None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:20.751298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a06f0-c1d7-43e9-9f4e-29dce47f0024 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:20.751784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a06f0-c1d7-43e9-9f4e-29dce47f0024 None admin] PATH_INFO: `/v3/projects/3322b00723974bd99e837d447bac22d1/users/295bb46627e043989e6dba2805607c99/roles/c255c186230d4339b46d5e1f7f1f6cbc` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:20.761320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 301/601] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:20.761970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1a893-1dc5-4660-81d3-ef534dc158a6 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:20.772438 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a06f0-c1d7-43e9-9f4e-29dce47f0024 None admin] RBAC: Authorizing `identity:create_grant(project_id=3322b00723974bd99e837d447bac22d1, user_id=295bb46627e043989e6dba2805607c99, role_id=c255c186230d4339b46d5e1f7f1f6cbc)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:20.774175 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a06f0-c1d7-43e9-9f4e-29dce47f0024 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:20.790853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 301/602] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:13:20 2022] PUT /identity/v3/projects/3322b00723974bd99e837d447bac22d1/users/295bb46627e043989e6dba2805607c99/roles/c255c186230d4339b46d5e1f7f1f6cbc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:13:20.792687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a06f0-c1d7-43e9-9f4e-29dce47f0024 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:20.872468 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1a893-1dc5-4660-81d3-ef534dc158a6 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:20.908005 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66001b57-c77e-4aa3-ac36-a196427e4737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:20.908784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66001b57-c77e-4aa3-ac36-a196427e4737 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:20.908986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66001b57-c77e-4aa3-ac36-a196427e4737 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:20.909267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66001b57-c77e-4aa3-ac36-a196427e4737 None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:20.910067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66001b57-c77e-4aa3-ac36-a196427e4737 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:20.910376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66001b57-c77e-4aa3-ac36-a196427e4737 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:20.911810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66001b57-c77e-4aa3-ac36-a196427e4737 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:20.926508 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 302/603] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:20.933653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66001b57-c77e-4aa3-ac36-a196427e4737 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:20.945138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a06f0-c1d7-43e9-9f4e-29dce47f0024 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:21.022222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edf55f-f7fa-4632-a86c-b9c5b1212fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:21.023428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edf55f-f7fa-4632-a86c-b9c5b1212fb2 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:21.023867 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edf55f-f7fa-4632-a86c-b9c5b1212fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:21.024335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edf55f-f7fa-4632-a86c-b9c5b1212fb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:21.045432 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edf55f-f7fa-4632-a86c-b9c5b1212fb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:21.046898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edf55f-f7fa-4632-a86c-b9c5b1212fb2 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-b9edf55f-f7fa-4632-a86c-b9c5b1212fb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:21.068477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:21.069595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:21.070655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:21.070655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:21.070655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:21.070655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:21.070655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:21.070655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:21.070655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:21.070655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:21.075222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 302/604] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:21.077733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edf55f-f7fa-4632-a86c-b9c5b1212fb2 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:21.097592 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66001b57-c77e-4aa3-ac36-a196427e4737 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:21.140051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7684c11-e3f2-42c5-82ed-f7cd42371daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:21.142547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7684c11-e3f2-42c5-82ed-f7cd42371daa None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:21.143742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7684c11-e3f2-42c5-82ed-f7cd42371daa None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:21.144892 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7684c11-e3f2-42c5-82ed-f7cd42371daa None admin] PATH_INFO: `/v3/projects/3322b00723974bd99e837d447bac22d1/users/72caf90f411f495bb22aa85df4d25eb3/roles/c255c186230d4339b46d5e1f7f1f6cbc` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:21.176806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7684c11-e3f2-42c5-82ed-f7cd42371daa None admin] RBAC: Authorizing `identity:create_grant(project_id=3322b00723974bd99e837d447bac22d1, user_id=72caf90f411f495bb22aa85df4d25eb3, role_id=c255c186230d4339b46d5e1f7f1f6cbc)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:21.178251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7684c11-e3f2-42c5-82ed-f7cd42371daa None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:21.210252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 303/605] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:13:20 2022] PUT /identity/v3/projects/3322b00723974bd99e837d447bac22d1/users/72caf90f411f495bb22aa85df4d25eb3/roles/c255c186230d4339b46d5e1f7f1f6cbc => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:13:21.246360 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edf55f-f7fa-4632-a86c-b9c5b1212fb2 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:21.285148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d444c-4857-4747-9f5d-7d8124bf5c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:21.286091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d444c-4857-4747-9f5d-7d8124bf5c12 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:21.286515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d444c-4857-4747-9f5d-7d8124bf5c12 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:21.286945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d444c-4857-4747-9f5d-7d8124bf5c12 None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:21.287869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d444c-4857-4747-9f5d-7d8124bf5c12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:21.288349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d444c-4857-4747-9f5d-7d8124bf5c12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:21.290189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d444c-4857-4747-9f5d-7d8124bf5c12 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:21.309327 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 303/606] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:21.316117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7684c11-e3f2-42c5-82ed-f7cd42371daa None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:21.330489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e4153-0ec2-4345-8985-94935c273c2b None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:21.332979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e4153-0ec2-4345-8985-94935c273c2b None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:21.333442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e4153-0ec2-4345-8985-94935c273c2b None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:21.335085 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 304/607] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:21.340868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea071ca9-37b2-4980-bff8-895093e0a173 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:21.341156 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea071ca9-37b2-4980-bff8-895093e0a173 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:21.341835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea071ca9-37b2-4980-bff8-895093e0a173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:21.369083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-ea071ca9-37b2-4980-bff8-895093e0a173 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:21.370328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-ea071ca9-37b2-4980-bff8-895093e0a173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:21.433856 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7684c11-e3f2-42c5-82ed-f7cd42371daa None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:21.475333 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2cb7b-f867-4546-8c79-e15d84e3de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:21.476561 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2cb7b-f867-4546-8c79-e15d84e3de4c None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:21.477020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2cb7b-f867-4546-8c79-e15d84e3de4c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:21.477436 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2cb7b-f867-4546-8c79-e15d84e3de4c None admin] PATH_INFO: `/v3/projects/3322b00723974bd99e837d447bac22d1/users/0d852975a2e84925a45775a16cc52f32/roles/c255c186230d4339b46d5e1f7f1f6cbc` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:21.504443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2cb7b-f867-4546-8c79-e15d84e3de4c None admin] RBAC: Authorizing `identity:create_grant(project_id=3322b00723974bd99e837d447bac22d1, user_id=0d852975a2e84925a45775a16cc52f32, role_id=c255c186230d4339b46d5e1f7f1f6cbc)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:21.505891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2cb7b-f867-4546-8c79-e15d84e3de4c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:21.512655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 305/608] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:13:21.529500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 304/609] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:13:21 2022] PUT /identity/v3/projects/3322b00723974bd99e837d447bac22d1/users/0d852975a2e84925a45775a16cc52f32/roles/c255c186230d4339b46d5e1f7f1f6cbc => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:13:21.540761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea071ca9-37b2-4980-bff8-895093e0a173 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:21.656505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea071ca9-37b2-4980-bff8-895093e0a173 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:21.673717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9bb5a-65aa-48e8-bbc6-db8a6780a2f5 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:21.673998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9bb5a-65aa-48e8-bbc6-db8a6780a2f5 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:21.674270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9bb5a-65aa-48e8-bbc6-db8a6780a2f5 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:21.675742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 305/610] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:13:21.681060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac0acc-1683-43c8-9d48-a4ab4fb1d42d None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:21.681545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac0acc-1683-43c8-9d48-a4ab4fb1d42d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:21.681965 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac0acc-1683-43c8-9d48-a4ab4fb1d42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:21.699490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c8ac0acc-1683-43c8-9d48-a4ab4fb1d42d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:21.701008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c8ac0acc-1683-43c8-9d48-a4ab4fb1d42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:21.711130 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d0210-cadd-4ddb-83ac-94fbf4794b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:21.712296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d0210-cadd-4ddb-83ac-94fbf4794b25 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:21.712592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d0210-cadd-4ddb-83ac-94fbf4794b25 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:21.712871 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d0210-cadd-4ddb-83ac-94fbf4794b25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:21.744503 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d0210-cadd-4ddb-83ac-94fbf4794b25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-7d1d0210-cadd-4ddb-83ac-94fbf4794b25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:21.754428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:21.755680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:21.756645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:21.756645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:21.756645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:21.756645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:21.756645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:21.756645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:21.756645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:21.756645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:21.756645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:21.756645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:21.756645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:21.756645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:21.760471 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 306/611] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:21.766916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d0210-cadd-4ddb-83ac-94fbf4794b25 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:21.840774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 306/612] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:13:21.863694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac0acc-1683-43c8-9d48-a4ab4fb1d42d None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:21.914623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d0210-cadd-4ddb-83ac-94fbf4794b25 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:21.949286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6cc6f-db67-4d57-9a45-c03a87b87830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:21.950108 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6cc6f-db67-4d57-9a45-c03a87b87830 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:21.950401 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6cc6f-db67-4d57-9a45-c03a87b87830 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:21.950741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6cc6f-db67-4d57-9a45-c03a87b87830 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:21.951799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6cc6f-db67-4d57-9a45-c03a87b87830 None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:21.952201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6cc6f-db67-4d57-9a45-c03a87b87830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:21.953936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6cc6f-db67-4d57-9a45-c03a87b87830 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:21.963062 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 307/613] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:21.968960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac0acc-1683-43c8-9d48-a4ab4fb1d42d None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:21.969128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6cc6f-db67-4d57-9a45-c03a87b87830 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:22.014151 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32001358-1fa1-47ec-a84f-ced0b559287e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:22.014943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32001358-1fa1-47ec-a84f-ced0b559287e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:22.015221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32001358-1fa1-47ec-a84f-ced0b559287e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:22.015492 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32001358-1fa1-47ec-a84f-ced0b559287e None admin] PATH_INFO: `/v3/roles/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:22.036605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32001358-1fa1-47ec-a84f-ced0b559287e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-32001358-1fa1-47ec-a84f-ced0b559287e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:22.044412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:22.046051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:22.046913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:22.046913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:22.046913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:22.046913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:22.046913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:22.046913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:22.046913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:22.046913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:22.046913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:22.046913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:22.046913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:22.046913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:22.052388 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 307/614] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:22.058786 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32001358-1fa1-47ec-a84f-ced0b559287e None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:22.080962 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6cc6f-db67-4d57-9a45-c03a87b87830 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:22.118466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697db83-3260-472e-a8e6-307444c48165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:22.119277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697db83-3260-472e-a8e6-307444c48165 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:22.119555 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697db83-3260-472e-a8e6-307444c48165 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:22.119825 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697db83-3260-472e-a8e6-307444c48165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:22.131639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697db83-3260-472e-a8e6-307444c48165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:22.133628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697db83-3260-472e-a8e6-307444c48165 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-4697db83-3260-472e-a8e6-307444c48165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:22.155088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:22.157373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:22.159517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:22.159517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:22.159517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:22.159517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:22.159517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 308/615] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:22.167428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697db83-3260-472e-a8e6-307444c48165 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:22.280382 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697db83-3260-472e-a8e6-307444c48165 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:22.298032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32001358-1fa1-47ec-a84f-ced0b559287e None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:22.316643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bab2ae-b007-4eeb-a966-02d6b3a6d8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:22.317360 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bab2ae-b007-4eeb-a966-02d6b3a6d8e7 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:22.318016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bab2ae-b007-4eeb-a966-02d6b3a6d8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:22.318306 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bab2ae-b007-4eeb-a966-02d6b3a6d8e7 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:22.319326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bab2ae-b007-4eeb-a966-02d6b3a6d8e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:22.319643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bab2ae-b007-4eeb-a966-02d6b3a6d8e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:22.321049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bab2ae-b007-4eeb-a966-02d6b3a6d8e7 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:22.335163 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 309/616] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:22.345699 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bab2ae-b007-4eeb-a966-02d6b3a6d8e7 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:22.359281 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e62004-3c61-4ef4-8a4f-830062e7b02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:22.360006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e62004-3c61-4ef4-8a4f-830062e7b02b None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:22.360292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e62004-3c61-4ef4-8a4f-830062e7b02b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:22.360564 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e62004-3c61-4ef4-8a4f-830062e7b02b None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:22.361315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e62004-3c61-4ef4-8a4f-830062e7b02b None admin] RBAC: Adding query filter params (name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:22.361618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e62004-3c61-4ef4-8a4f-830062e7b02b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:22.362989 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e62004-3c61-4ef4-8a4f-830062e7b02b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:22.371447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 308/617] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:22.377652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e62004-3c61-4ef4-8a4f-830062e7b02b None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:22.517466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bab2ae-b007-4eeb-a966-02d6b3a6d8e7 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:22.566394 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e62004-3c61-4ef4-8a4f-830062e7b02b None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:22.582558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a4c4c-295b-4286-9420-1dc442df1f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:22.583460 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a4c4c-295b-4286-9420-1dc442df1f2f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:22.583745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a4c4c-295b-4286-9420-1dc442df1f2f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:22.584045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a4c4c-295b-4286-9420-1dc442df1f2f None admin] PATH_INFO: `/v3/users/nova` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:22.593915 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a4c4c-295b-4286-9420-1dc442df1f2f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:22.595353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a4c4c-295b-4286-9420-1dc442df1f2f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-953a4c4c-295b-4286-9420-1dc442df1f2f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:22.618209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:22.619196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:22.620164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:22.620164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:22.620164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:22.620164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:22.620164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:22.620164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:22.620164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:22.620164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:22.620164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 04:13:22.620164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:22.624042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 310/618] 10.222.0.44 () {56 vars in 1093 bytes} [Mon May 9 04:13:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:22.625756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a4c4c-295b-4286-9420-1dc442df1f2f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:22.652588 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65ff17-2742-4a6d-baf3-0cd1522e69a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:22.653772 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65ff17-2742-4a6d-baf3-0cd1522e69a1 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:22.655014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65ff17-2742-4a6d-baf3-0cd1522e69a1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:22.655501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65ff17-2742-4a6d-baf3-0cd1522e69a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:22.676195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65ff17-2742-4a6d-baf3-0cd1522e69a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:22.677979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65ff17-2742-4a6d-baf3-0cd1522e69a1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-8f65ff17-2742-4a6d-baf3-0cd1522e69a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:22.715918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:22.717413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:22.718439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:22.718439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:22.718439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:22.718439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:22.720797 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 309/619] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:22.725186 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a4c4c-295b-4286-9420-1dc442df1f2f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:22.726582 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65ff17-2742-4a6d-baf3-0cd1522e69a1 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:22.759835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0ff24-6dbf-4512-866b-3d082dee4a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:22.760731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0ff24-6dbf-4512-866b-3d082dee4a1f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:22.761017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0ff24-6dbf-4512-866b-3d082dee4a1f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:22.761288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0ff24-6dbf-4512-866b-3d082dee4a1f None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:22.762118 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0ff24-6dbf-4512-866b-3d082dee4a1f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:22.762420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0ff24-6dbf-4512-866b-3d082dee4a1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:22.763869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0ff24-6dbf-4512-866b-3d082dee4a1f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:22.794377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 311/620] 10.222.0.44 () {56 vars in 1133 bytes} [Mon May 9 04:13:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:22.803160 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0ff24-6dbf-4512-866b-3d082dee4a1f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:22.902951 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65ff17-2742-4a6d-baf3-0cd1522e69a1 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:22.935746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0ff24-6dbf-4512-866b-3d082dee4a1f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:22.947383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cf55a-1999-444a-8139-3d8d8e2f44f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:22.948092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cf55a-1999-444a-8139-3d8d8e2f44f4 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:22.948375 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cf55a-1999-444a-8139-3d8d8e2f44f4 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:22.948659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cf55a-1999-444a-8139-3d8d8e2f44f4 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:22.949381 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cf55a-1999-444a-8139-3d8d8e2f44f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:22.949687 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cf55a-1999-444a-8139-3d8d8e2f44f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:22.951336 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cf55a-1999-444a-8139-3d8d8e2f44f4 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:22.972636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97e5b9-0f48-465a-a2c1-10c6d460d1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:22.973666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97e5b9-0f48-465a-a2c1-10c6d460d1da None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:22.974206 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97e5b9-0f48-465a-a2c1-10c6d460d1da None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:22.974678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97e5b9-0f48-465a-a2c1-10c6d460d1da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:22.978144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 310/621] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:22.984297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cf55a-1999-444a-8139-3d8d8e2f44f4 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:23.066202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97e5b9-0f48-465a-a2c1-10c6d460d1da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:23.068034 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97e5b9-0f48-465a-a2c1-10c6d460d1da None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:23.177104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 312/622] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:13:23.184923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97e5b9-0f48-465a-a2c1-10c6d460d1da None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:23.228583 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cf55a-1999-444a-8139-3d8d8e2f44f4 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:23.293363 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5fb0e-8df4-42a7-a2d5-b91c0fb9171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:23.294220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5fb0e-8df4-42a7-a2d5-b91c0fb9171f None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:23.294585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5fb0e-8df4-42a7-a2d5-b91c0fb9171f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:23.294854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5fb0e-8df4-42a7-a2d5-b91c0fb9171f None admin] PATH_INFO: `/v3/users/swift` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:23.307145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5fb0e-8df4-42a7-a2d5-b91c0fb9171f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:23.308613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5fb0e-8df4-42a7-a2d5-b91c0fb9171f None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-86e5fb0e-8df4-42a7-a2d5-b91c0fb9171f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:23.330928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:23.331913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:23.332850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:23.332850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:23.332850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:23.332850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:23.332850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:23.332850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:23.332850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:23.332850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:23.332850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 04:13:23.332850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:23.340376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 311/623] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:13:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:23.347566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5fb0e-8df4-42a7-a2d5-b91c0fb9171f None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:23.353187 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97e5b9-0f48-465a-a2c1-10c6d460d1da None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:23.417545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157951d-1bd6-4d97-a987-46a415cc1fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:23.418595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157951d-1bd6-4d97-a987-46a415cc1fdd None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:23.418900 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157951d-1bd6-4d97-a987-46a415cc1fdd None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:23.419176 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157951d-1bd6-4d97-a987-46a415cc1fdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:23.432051 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157951d-1bd6-4d97-a987-46a415cc1fdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:23.433426 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157951d-1bd6-4d97-a987-46a415cc1fdd None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-9157951d-1bd6-4d97-a987-46a415cc1fdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:23.456325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:23.457277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:23.458148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:23.458148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:23.458148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:23.458148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:23.458148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 313/624] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:23.459731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553402c8-e2de-4d59-9b68-0cbb779f23fa None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:23.459731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553402c8-e2de-4d59-9b68-0cbb779f23fa None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:23.459731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553402c8-e2de-4d59-9b68-0cbb779f23fa None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:23.463967 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 314/625] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:23.464973 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553402c8-e2de-4d59-9b68-0cbb779f23fa None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:23.470744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5fb0e-8df4-42a7-a2d5-b91c0fb9171f None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:23.505273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d29b99-11db-4d05-9fbb-6f6a2f3d3dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:23.505943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d29b99-11db-4d05-9fbb-6f6a2f3d3dd9 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:23.506225 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d29b99-11db-4d05-9fbb-6f6a2f3d3dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:23.506501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d29b99-11db-4d05-9fbb-6f6a2f3d3dd9 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:23.507404 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d29b99-11db-4d05-9fbb-6f6a2f3d3dd9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:23.507714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d29b99-11db-4d05-9fbb-6f6a2f3d3dd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:23.509235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d29b99-11db-4d05-9fbb-6f6a2f3d3dd9 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:23.527658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 312/626] 10.222.0.44 () {56 vars in 1135 bytes} [Mon May 9 04:13:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:23.529037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed787f3-6dd9-42a3-ab59-1c114ed17ccf None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:23.529376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed787f3-6dd9-42a3-ab59-1c114ed17ccf None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:23.529652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed787f3-6dd9-42a3-ab59-1c114ed17ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:23.546210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-1ed787f3-6dd9-42a3-ab59-1c114ed17ccf None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:23.547455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1ed787f3-6dd9-42a3-ab59-1c114ed17ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:23.571704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553402c8-e2de-4d59-9b68-0cbb779f23fa None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:23.609307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4819c7-59d2-4b99-89d7-2ec2b4c1164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:23.610380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4819c7-59d2-4b99-89d7-2ec2b4c1164c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:23.610875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4819c7-59d2-4b99-89d7-2ec2b4c1164c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:23.611370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4819c7-59d2-4b99-89d7-2ec2b4c1164c None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:23.612341 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4819c7-59d2-4b99-89d7-2ec2b4c1164c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:23.612861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4819c7-59d2-4b99-89d7-2ec2b4c1164c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:23.614730 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4819c7-59d2-4b99-89d7-2ec2b4c1164c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:23.624499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 313/627] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:23.625152 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed787f3-6dd9-42a3-ab59-1c114ed17ccf None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:23.628314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 315/628] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:23.639465 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4819c7-59d2-4b99-89d7-2ec2b4c1164c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:23.735473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed787f3-6dd9-42a3-ab59-1c114ed17ccf None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:23.765338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4819c7-59d2-4b99-89d7-2ec2b4c1164c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:23.787195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbda4e-3e00-4bf2-b2fe-4651f7b049c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:23.788018 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbda4e-3e00-4bf2-b2fe-4651f7b049c7 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:23.788298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbda4e-3e00-4bf2-b2fe-4651f7b049c7 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:23.788567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbda4e-3e00-4bf2-b2fe-4651f7b049c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:23.799287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbda4e-3e00-4bf2-b2fe-4651f7b049c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:23.800938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbda4e-3e00-4bf2-b2fe-4651f7b049c7 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:23.804789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e954fa-52a8-470a-910d-dab53ceccb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:23.805591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e954fa-52a8-470a-910d-dab53ceccb9e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:23.805875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e954fa-52a8-470a-910d-dab53ceccb9e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:23.806154 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e954fa-52a8-470a-910d-dab53ceccb9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:23.814608 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e954fa-52a8-470a-910d-dab53ceccb9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:23.816054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e954fa-52a8-470a-910d-dab53ceccb9e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-08bbda4e-3e00-4bf2-b2fe-4651f7b049c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:23.819815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:23.820705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:23.822378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:23.822378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:23.822378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:23.822378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:23.822378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 314/629] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:23.822378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbda4e-3e00-4bf2-b2fe-4651f7b049c7 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-e4e954fa-52a8-470a-910d-dab53ceccb9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:23.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:23.833331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:23.834357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:23.834357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:23.834357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:23.834357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:23.834357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:23.834357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:23.834357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:23.834357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:23.834357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 316/630] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:23.838713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e954fa-52a8-470a-910d-dab53ceccb9e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:23.924848 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbda4e-3e00-4bf2-b2fe-4651f7b049c7 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:23.962054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e954fa-52a8-470a-910d-dab53ceccb9e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:23.962620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823008d4-8089-4338-ad06-e2a7793388be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:23.963571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823008d4-8089-4338-ad06-e2a7793388be None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:23.963995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823008d4-8089-4338-ad06-e2a7793388be None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:23.964373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823008d4-8089-4338-ad06-e2a7793388be None admin] PATH_INFO: `/v3/roles/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:23.976850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823008d4-8089-4338-ad06-e2a7793388be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-823008d4-8089-4338-ad06-e2a7793388be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:23.985299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:23.987258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:23.989257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:23.989257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:23.989257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:23.989257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:23.989257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:23.989257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:23.989257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:23.989257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:23.989257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:23.989257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:23.989257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:23.989257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:23.999174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 315/631] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:23.999174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823008d4-8089-4338-ad06-e2a7793388be None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:24.002905 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23656bfb-0e47-46ba-9758-1d243c9a84a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:24.003774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23656bfb-0e47-46ba-9758-1d243c9a84a3 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.004388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23656bfb-0e47-46ba-9758-1d243c9a84a3 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.004587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23656bfb-0e47-46ba-9758-1d243c9a84a3 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.007029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23656bfb-0e47-46ba-9758-1d243c9a84a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:24.007345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23656bfb-0e47-46ba-9758-1d243c9a84a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:24.008743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23656bfb-0e47-46ba-9758-1d243c9a84a3 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:24.020530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 317/632] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:24.021437 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23656bfb-0e47-46ba-9758-1d243c9a84a3 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:24.115782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823008d4-8089-4338-ad06-e2a7793388be None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:24.121624 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23656bfb-0e47-46ba-9758-1d243c9a84a3 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:24.149372 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15b5f2-969e-4dd8-bfc6-bac9991cb3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:24.150029 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15b5f2-969e-4dd8-bfc6-bac9991cb3a4 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.150308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15b5f2-969e-4dd8-bfc6-bac9991cb3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.150624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15b5f2-969e-4dd8-bfc6-bac9991cb3a4 None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.151417 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15b5f2-969e-4dd8-bfc6-bac9991cb3a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:24.151843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15b5f2-969e-4dd8-bfc6-bac9991cb3a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:24.153483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15b5f2-969e-4dd8-bfc6-bac9991cb3a4 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:24.155157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ef6e6-2d02-41bc-99f2-5af141e8e705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:24.155822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ef6e6-2d02-41bc-99f2-5af141e8e705 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.156131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ef6e6-2d02-41bc-99f2-5af141e8e705 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.156410 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ef6e6-2d02-41bc-99f2-5af141e8e705 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.157153 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ef6e6-2d02-41bc-99f2-5af141e8e705 None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:24.157458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ef6e6-2d02-41bc-99f2-5af141e8e705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:24.158805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ef6e6-2d02-41bc-99f2-5af141e8e705 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:24.166443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 316/633] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:24.166934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 318/634] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:24.167316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15b5f2-969e-4dd8-bfc6-bac9991cb3a4 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:24.168124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ded12-a821-4335-a6ec-5ea3492b1761 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.168457 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ded12-a821-4335-a6ec-5ea3492b1761 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.168735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ded12-a821-4335-a6ec-5ea3492b1761 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.174323 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 319/635] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:24.175558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddbd62-5059-4943-b79d-ab6dbb66d0f8 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.175680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddbd62-5059-4943-b79d-ab6dbb66d0f8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.175983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddbd62-5059-4943-b79d-ab6dbb66d0f8 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.177244 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 320/636] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:24.178198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddbd62-5059-4943-b79d-ab6dbb66d0f8 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:24.260066 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15b5f2-969e-4dd8-bfc6-bac9991cb3a4 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:24.273074 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddbd62-5059-4943-b79d-ab6dbb66d0f8 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:24.291300 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df27e1d-ba1e-4c1a-ae2c-cb0aac32a003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:24.292109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df27e1d-ba1e-4c1a-ae2c-cb0aac32a003 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.292388 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df27e1d-ba1e-4c1a-ae2c-cb0aac32a003 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.292657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df27e1d-ba1e-4c1a-ae2c-cb0aac32a003 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.299011 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df27e1d-ba1e-4c1a-ae2c-cb0aac32a003 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:24.300459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df27e1d-ba1e-4c1a-ae2c-cb0aac32a003 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:24.303810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab45e83-9254-4a56-8ab7-c7b283014b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:24.304481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab45e83-9254-4a56-8ab7-c7b283014b12 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.304764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab45e83-9254-4a56-8ab7-c7b283014b12 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.305041 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab45e83-9254-4a56-8ab7-c7b283014b12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.305861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab45e83-9254-4a56-8ab7-c7b283014b12 None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=faf6107735424ff7901614953e7df403) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:24.306172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab45e83-9254-4a56-8ab7-c7b283014b12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:24.307552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab45e83-9254-4a56-8ab7-c7b283014b12 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:24.313724 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 321/637] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:24 2022] GET /identity/v3/role_assignments?user.id=faf6107735424ff7901614953e7df403&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-0df27e1d-ba1e-4c1a-ae2c-cb0aac32a003 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:24.314160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:24.315342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:24.316507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:24.316507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:24.316507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:24.316507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:24.316507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:24.316507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:24.316507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:24.316507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:24.316928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709fdf8-89ba-41c0-88a4-627f5a5818f8 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.316928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709fdf8-89ba-41c0-88a4-627f5a5818f8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.316928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709fdf8-89ba-41c0-88a4-627f5a5818f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.320208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 317/638] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:24.321003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df27e1d-ba1e-4c1a-ae2c-cb0aac32a003 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:24.334063 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d709fdf8-89ba-41c0-88a4-627f5a5818f8 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:24.335295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d709fdf8-89ba-41c0-88a4-627f5a5818f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:24.415922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 322/639] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:24.415922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d8994-609c-4776-badf-440972852d7d None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.415922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d8994-609c-4776-badf-440972852d7d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.415922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d8994-609c-4776-badf-440972852d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.434231 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df27e1d-ba1e-4c1a-ae2c-cb0aac32a003 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:24.436822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-774d8994-609c-4776-badf-440972852d7d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:24.438039 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-774d8994-609c-4776-badf-440972852d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:24.469994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbba935-aa70-4067-a60c-b631ab354887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:24.471101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbba935-aa70-4067-a60c-b631ab354887 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.471681 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbba935-aa70-4067-a60c-b631ab354887 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.472170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbba935-aa70-4067-a60c-b631ab354887 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.484178 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbba935-aa70-4067-a60c-b631ab354887 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:24.486655 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbba935-aa70-4067-a60c-b631ab354887 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:24.505682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 323/640] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:24.506749 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d8994-609c-4776-badf-440972852d7d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-3dbba935-aa70-4067-a60c-b631ab354887 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:24.508429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:24.509477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:24.510588 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:24.510588 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:24.510588 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:24.510588 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:24.516687 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 318/641] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:24.517813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbba935-aa70-4067-a60c-b631ab354887 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:24.625942 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d8994-609c-4776-badf-440972852d7d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:24.626458 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbba935-aa70-4067-a60c-b631ab354887 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:24.659907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b506892-ebf3-4bcf-8704-bf73487ca974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:24.660685 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b506892-ebf3-4bcf-8704-bf73487ca974 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.661040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b506892-ebf3-4bcf-8704-bf73487ca974 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.661388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b506892-ebf3-4bcf-8704-bf73487ca974 None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.662232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16dd42-a9f7-4297-bd2a-bc0a4b08d66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:24.662346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b506892-ebf3-4bcf-8704-bf73487ca974 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:24.662661 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b506892-ebf3-4bcf-8704-bf73487ca974 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:24.663032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16dd42-a9f7-4297-bd2a-bc0a4b08d66a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.663316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16dd42-a9f7-4297-bd2a-bc0a4b08d66a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.663591 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16dd42-a9f7-4297-bd2a-bc0a4b08d66a None admin] PATH_INFO: `/v3/roles/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.664378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b506892-ebf3-4bcf-8704-bf73487ca974 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:24.673844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16dd42-a9f7-4297-bd2a-bc0a4b08d66a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:24.680203 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 324/642] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:24.680203 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b506892-ebf3-4bcf-8704-bf73487ca974 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-7c16dd42-a9f7-4297-bd2a-bc0a4b08d66a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:24.681658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:24.682844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 319/643] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:24.684016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16dd42-a9f7-4297-bd2a-bc0a4b08d66a None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:24.785059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16dd42-a9f7-4297-bd2a-bc0a4b08d66a None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:24.791115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b506892-ebf3-4bcf-8704-bf73487ca974 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:24.818062 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165288b-1b10-4f4a-8b3b-3e2b70ab1ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:24.818866 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165288b-1b10-4f4a-8b3b-3e2b70ab1ba0 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.819144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165288b-1b10-4f4a-8b3b-3e2b70ab1ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.819421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165288b-1b10-4f4a-8b3b-3e2b70ab1ba0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.824296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c624261-5413-4972-9dea-51406897f1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:24.825142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c624261-5413-4972-9dea-51406897f1c0 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.825527 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c624261-5413-4972-9dea-51406897f1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.826482 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c624261-5413-4972-9dea-51406897f1c0 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.827705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c624261-5413-4972-9dea-51406897f1c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:24.828126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c624261-5413-4972-9dea-51406897f1c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:24.829809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c624261-5413-4972-9dea-51406897f1c0 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:24.831381 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165288b-1b10-4f4a-8b3b-3e2b70ab1ba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-a165288b-1b10-4f4a-8b3b-3e2b70ab1ba0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:24.839594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:24.840594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:24.841515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:24.841515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:24.841515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:24.841515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:24.841515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:24.841515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:24.841515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:24.841515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:24.841515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:24.841515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:24.841515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:24.841515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:24.841515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 320/644] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:24.842808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165288b-1b10-4f4a-8b3b-3e2b70ab1ba0 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:24.846284 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 325/645] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:24.847366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c624261-5413-4972-9dea-51406897f1c0 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:24.946289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c624261-5413-4972-9dea-51406897f1c0 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:24.953240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165288b-1b10-4f4a-8b3b-3e2b70ab1ba0 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:24.977266 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb902b2e-3320-491c-bd3f-370249b8e414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:24.977931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb902b2e-3320-491c-bd3f-370249b8e414 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.978217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb902b2e-3320-491c-bd3f-370249b8e414 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.978488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb902b2e-3320-491c-bd3f-370249b8e414 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.979224 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb902b2e-3320-491c-bd3f-370249b8e414 None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:24.979521 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb902b2e-3320-491c-bd3f-370249b8e414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:24.980919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb902b2e-3320-491c-bd3f-370249b8e414 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:24.986257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f21e4-8c12-417b-8c34-ee172a65a858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:24.987364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f21e4-8c12-417b-8c34-ee172a65a858 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:24.987842 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f21e4-8c12-417b-8c34-ee172a65a858 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:24.988336 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f21e4-8c12-417b-8c34-ee172a65a858 None admin] PATH_INFO: `/v3/projects/3322b00723974bd99e837d447bac22d1/users/c46e7cc6407f44f08a659288e3ad0767/roles/c255c186230d4339b46d5e1f7f1f6cbc` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:24.988837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 326/646] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:24.989619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb902b2e-3320-491c-bd3f-370249b8e414 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:25.008564 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f21e4-8c12-417b-8c34-ee172a65a858 None admin] RBAC: Authorizing `identity:create_grant(project_id=3322b00723974bd99e837d447bac22d1, user_id=c46e7cc6407f44f08a659288e3ad0767, role_id=c255c186230d4339b46d5e1f7f1f6cbc)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:25.010001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f21e4-8c12-417b-8c34-ee172a65a858 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:25.031926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 321/647] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:13:24 2022] PUT /identity/v3/projects/3322b00723974bd99e837d447bac22d1/users/c46e7cc6407f44f08a659288e3ad0767/roles/c255c186230d4339b46d5e1f7f1f6cbc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:13:25.031926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f21e4-8c12-417b-8c34-ee172a65a858 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:25.113860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb902b2e-3320-491c-bd3f-370249b8e414 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:25.155281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44b465-78ca-4769-a398-ac0c4d5927e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:25.156151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44b465-78ca-4769-a398-ac0c4d5927e6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:25.156502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44b465-78ca-4769-a398-ac0c4d5927e6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:25.156842 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed44b465-78ca-4769-a398-ac0c4d5927e6 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:25.157733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44b465-78ca-4769-a398-ac0c4d5927e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:25.158102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44b465-78ca-4769-a398-ac0c4d5927e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:25.159556 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed44b465-78ca-4769-a398-ac0c4d5927e6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:25.164679 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f21e4-8c12-417b-8c34-ee172a65a858 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:25.168806 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 327/648] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:25.170343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44b465-78ca-4769-a398-ac0c4d5927e6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:25.210885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbffe8-2acf-4485-bb5f-043eeb070ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:25.213084 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbffe8-2acf-4485-bb5f-043eeb070ab4 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:25.213084 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbffe8-2acf-4485-bb5f-043eeb070ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:25.213084 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbffe8-2acf-4485-bb5f-043eeb070ab4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:25.220271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbffe8-2acf-4485-bb5f-043eeb070ab4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:25.221719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbffe8-2acf-4485-bb5f-043eeb070ab4 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-d8bbffe8-2acf-4485-bb5f-043eeb070ab4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:25.240645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:25.242164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:25.243111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:25.243111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:25.243111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:25.243111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:25.243111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:25.243111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:25.243111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:25.243111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:25.243111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 04:13:25.243111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:25.243111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 322/649] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:25.243111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbffe8-2acf-4485-bb5f-043eeb070ab4 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:25.300234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed44b465-78ca-4769-a398-ac0c4d5927e6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:25.342853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87537d04-4ea7-43fa-899e-f18ee30b6c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:25.343693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87537d04-4ea7-43fa-899e-f18ee30b6c42 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:25.344007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87537d04-4ea7-43fa-899e-f18ee30b6c42 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:25.344287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87537d04-4ea7-43fa-899e-f18ee30b6c42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:25.355116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbffe8-2acf-4485-bb5f-043eeb070ab4 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:25.358368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87537d04-4ea7-43fa-899e-f18ee30b6c42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:25.360706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87537d04-4ea7-43fa-899e-f18ee30b6c42 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-87537d04-4ea7-43fa-899e-f18ee30b6c42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:25.382384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:25.383368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:25.384316 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:25.384316 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:25.384316 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:25.384316 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:25.384316 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 328/650] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:25.388793 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87537d04-4ea7-43fa-899e-f18ee30b6c42 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:25.398251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d29fa-4d05-41c4-b306-d9933e02f88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:25.399053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d29fa-4d05-41c4-b306-d9933e02f88d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:25.399335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d29fa-4d05-41c4-b306-d9933e02f88d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:25.399612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d29fa-4d05-41c4-b306-d9933e02f88d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:25.410488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d29fa-4d05-41c4-b306-d9933e02f88d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:25.411959 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d29fa-4d05-41c4-b306-d9933e02f88d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-af5d29fa-4d05-41c4-b306-d9933e02f88d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:25.433068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:25.433985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:25.434883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:25.434883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:25.434883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:25.434883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:25.434883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 323/651] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:25.436267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d29fa-4d05-41c4-b306-d9933e02f88d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:25.521305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87537d04-4ea7-43fa-899e-f18ee30b6c42 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:25.538087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d29fa-4d05-41c4-b306-d9933e02f88d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:25.564397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c6260-986b-4fe3-acd7-d085cbc6ac98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:25.565585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c6260-986b-4fe3-acd7-d085cbc6ac98 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:25.566196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c6260-986b-4fe3-acd7-d085cbc6ac98 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:25.566668 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c6260-986b-4fe3-acd7-d085cbc6ac98 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:25.567952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c6260-986b-4fe3-acd7-d085cbc6ac98 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:25.568796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c6260-986b-4fe3-acd7-d085cbc6ac98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:25.570676 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c6260-986b-4fe3-acd7-d085cbc6ac98 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:25.583859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623dc13-4fe9-448f-9f6d-cc2fc4b8d1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:25.584912 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623dc13-4fe9-448f-9f6d-cc2fc4b8d1de None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:25.585327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623dc13-4fe9-448f-9f6d-cc2fc4b8d1de None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:25.585743 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623dc13-4fe9-448f-9f6d-cc2fc4b8d1de None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:25.586912 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623dc13-4fe9-448f-9f6d-cc2fc4b8d1de None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:25.587385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623dc13-4fe9-448f-9f6d-cc2fc4b8d1de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:25.590225 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623dc13-4fe9-448f-9f6d-cc2fc4b8d1de None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:25.595212 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 329/652] 10.222.0.44 () {56 vars in 1137 bytes} [Mon May 9 04:13:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:25.596499 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c6260-986b-4fe3-acd7-d085cbc6ac98 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:25.610637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 324/653] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:25.610637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623dc13-4fe9-448f-9f6d-cc2fc4b8d1de None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:25.720070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623dc13-4fe9-448f-9f6d-cc2fc4b8d1de None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:25.722983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c6260-986b-4fe3-acd7-d085cbc6ac98 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:25.754665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21023bc0-bd9f-41bf-8d64-61a60a0960f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:25.755321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21023bc0-bd9f-41bf-8d64-61a60a0960f2 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:25.755607 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21023bc0-bd9f-41bf-8d64-61a60a0960f2 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:25.755880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21023bc0-bd9f-41bf-8d64-61a60a0960f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:25.769713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1791573-8f8f-46f8-9595-15af1949b757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:25.770927 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1791573-8f8f-46f8-9595-15af1949b757 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:25.771435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1791573-8f8f-46f8-9595-15af1949b757 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:25.772060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1791573-8f8f-46f8-9595-15af1949b757 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:25.773640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1791573-8f8f-46f8-9595-15af1949b757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:25.774147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1791573-8f8f-46f8-9595-15af1949b757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:25.776189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1791573-8f8f-46f8-9595-15af1949b757 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:25.792079 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 330/654] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:25.793199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1791573-8f8f-46f8-9595-15af1949b757 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:25.833142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21023bc0-bd9f-41bf-8d64-61a60a0960f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:25.835344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21023bc0-bd9f-41bf-8d64-61a60a0960f2 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:25.893803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1791573-8f8f-46f8-9595-15af1949b757 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:25.938143 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 325/655] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:13:25.939659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21023bc0-bd9f-41bf-8d64-61a60a0960f2 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:25.943759 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70297aa3-db67-462a-86ae-5b3773d7a25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:25.943931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70297aa3-db67-462a-86ae-5b3773d7a25d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:25.944312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70297aa3-db67-462a-86ae-5b3773d7a25d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:25.944686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70297aa3-db67-462a-86ae-5b3773d7a25d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:25.952849 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70297aa3-db67-462a-86ae-5b3773d7a25d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:25.954646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70297aa3-db67-462a-86ae-5b3773d7a25d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-70297aa3-db67-462a-86ae-5b3773d7a25d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:25.975560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:25.976674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:25.977620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:25.977620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:25.977620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:25.977620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:25.977620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:25.977620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:25.977620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:25.977620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:25.977620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 04:13:25.977620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:25.982765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 331/656] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:25.984054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70297aa3-db67-462a-86ae-5b3773d7a25d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:26.041185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21023bc0-bd9f-41bf-8d64-61a60a0960f2 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:26.075424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b56fb0-83ab-43e6-89d1-4391226c5dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:26.076630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b56fb0-83ab-43e6-89d1-4391226c5dce None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:26.077132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b56fb0-83ab-43e6-89d1-4391226c5dce None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:26.077598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b56fb0-83ab-43e6-89d1-4391226c5dce None admin] PATH_INFO: `/v3/users/neutron` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:26.085207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b56fb0-83ab-43e6-89d1-4391226c5dce None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:26.086891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b56fb0-83ab-43e6-89d1-4391226c5dce None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:26.088113 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70297aa3-db67-462a-86ae-5b3773d7a25d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-b6b56fb0-83ab-43e6-89d1-4391226c5dce None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:26.101831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:26.102769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:26.103709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:26.103709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:26.103709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:26.103709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:26.103709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:26.103709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:26.103709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:26.103709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:26.103709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 04:13:26.103709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:26.106289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 326/657] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:26.107773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b56fb0-83ab-43e6-89d1-4391226c5dce None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:26.122441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1ac8d-453f-4575-8cc9-d44d3dc2eb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:26.124066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1ac8d-453f-4575-8cc9-d44d3dc2eb8a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:26.124066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1ac8d-453f-4575-8cc9-d44d3dc2eb8a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:26.124357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1ac8d-453f-4575-8cc9-d44d3dc2eb8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:26.135135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1ac8d-453f-4575-8cc9-d44d3dc2eb8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:26.136737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1ac8d-453f-4575-8cc9-d44d3dc2eb8a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-caf1ac8d-453f-4575-8cc9-d44d3dc2eb8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:26.154179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:26.155202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:26.156099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:26.156099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:26.156099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:26.156099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:26.156099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 332/658] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:26.156801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1ac8d-453f-4575-8cc9-d44d3dc2eb8a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:26.211638 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b56fb0-83ab-43e6-89d1-4391226c5dce None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:26.250108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525d7eb-d0aa-4e69-9165-bc7cb4caa779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:26.251190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525d7eb-d0aa-4e69-9165-bc7cb4caa779 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:26.251477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525d7eb-d0aa-4e69-9165-bc7cb4caa779 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:26.251750 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525d7eb-d0aa-4e69-9165-bc7cb4caa779 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:26.252789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525d7eb-d0aa-4e69-9165-bc7cb4caa779 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:26.253105 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525d7eb-d0aa-4e69-9165-bc7cb4caa779 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:26.255132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525d7eb-d0aa-4e69-9165-bc7cb4caa779 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:26.267088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1ac8d-453f-4575-8cc9-d44d3dc2eb8a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:26.274343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 327/659] 10.222.0.44 () {56 vars in 1137 bytes} [Mon May 9 04:13:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:26.275272 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525d7eb-d0aa-4e69-9165-bc7cb4caa779 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:26.314220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ddabd-818c-4869-880c-fbf3027ab702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:26.316107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ddabd-818c-4869-880c-fbf3027ab702 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:26.316600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ddabd-818c-4869-880c-fbf3027ab702 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:26.317076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ddabd-818c-4869-880c-fbf3027ab702 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:26.318477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ddabd-818c-4869-880c-fbf3027ab702 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:26.319330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ddabd-818c-4869-880c-fbf3027ab702 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:26.321434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ddabd-818c-4869-880c-fbf3027ab702 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:26.346011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 333/660] 10.222.0.44 () {56 vars in 1139 bytes} [Mon May 9 04:13:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:26.347972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ddabd-818c-4869-880c-fbf3027ab702 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:26.383744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525d7eb-d0aa-4e69-9165-bc7cb4caa779 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:26.416576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64ea34-8458-4e2d-a2ef-f9a2b77198eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:26.417237 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64ea34-8458-4e2d-a2ef-f9a2b77198eb None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:26.417558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64ea34-8458-4e2d-a2ef-f9a2b77198eb None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:26.417843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64ea34-8458-4e2d-a2ef-f9a2b77198eb None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:26.418555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64ea34-8458-4e2d-a2ef-f9a2b77198eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:26.418945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64ea34-8458-4e2d-a2ef-f9a2b77198eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:26.420914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64ea34-8458-4e2d-a2ef-f9a2b77198eb None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:26.432327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 328/661] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:26.433552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64ea34-8458-4e2d-a2ef-f9a2b77198eb None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:26.473555 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ddabd-818c-4869-880c-fbf3027ab702 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:26.513368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ea94b-6213-48d5-80d6-479a18ca53e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:26.514077 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ea94b-6213-48d5-80d6-479a18ca53e5 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:26.514362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ea94b-6213-48d5-80d6-479a18ca53e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:26.514634 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ea94b-6213-48d5-80d6-479a18ca53e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:26.539687 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64ea34-8458-4e2d-a2ef-f9a2b77198eb None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:26.580716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ea94b-6213-48d5-80d6-479a18ca53e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:26.582200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ea94b-6213-48d5-80d6-479a18ca53e5 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:26.582797 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda8f82-68f0-4db0-bd55-6e0569f97a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:26.583696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda8f82-68f0-4db0-bd55-6e0569f97a33 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:26.584430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda8f82-68f0-4db0-bd55-6e0569f97a33 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:26.584946 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda8f82-68f0-4db0-bd55-6e0569f97a33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:26.669461 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda8f82-68f0-4db0-bd55-6e0569f97a33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:26.671360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda8f82-68f0-4db0-bd55-6e0569f97a33 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:26.687773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 334/662] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:13:26.688754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ea94b-6213-48d5-80d6-479a18ca53e5 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:26.771692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 329/663] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 04:13:26.772840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda8f82-68f0-4db0-bd55-6e0569f97a33 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:26.799492 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ea94b-6213-48d5-80d6-479a18ca53e5 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:26.833653 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f713438-0280-44a7-b2aa-ed6b7ec87a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:26.834455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f713438-0280-44a7-b2aa-ed6b7ec87a0c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:26.834734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f713438-0280-44a7-b2aa-ed6b7ec87a0c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:26.835164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f713438-0280-44a7-b2aa-ed6b7ec87a0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:26.843790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f713438-0280-44a7-b2aa-ed6b7ec87a0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:26.845270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f713438-0280-44a7-b2aa-ed6b7ec87a0c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-4f713438-0280-44a7-b2aa-ed6b7ec87a0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:26.858662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:26.859545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:26.860799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:26.860799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:26.860799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:26.860799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:26.860799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:26.860799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:26.860799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:26.860799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:26.860799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 335/664] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:26.861285 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f713438-0280-44a7-b2aa-ed6b7ec87a0c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:26.872938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda8f82-68f0-4db0-bd55-6e0569f97a33 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:26.905285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b84d2-af11-47ec-8e65-4ff2d62ca6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:26.906090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b84d2-af11-47ec-8e65-4ff2d62ca6b0 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:26.906367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b84d2-af11-47ec-8e65-4ff2d62ca6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:26.906638 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b84d2-af11-47ec-8e65-4ff2d62ca6b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:26.916311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b84d2-af11-47ec-8e65-4ff2d62ca6b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:26.917737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b84d2-af11-47ec-8e65-4ff2d62ca6b0 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-930b84d2-af11-47ec-8e65-4ff2d62ca6b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:26.937740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:26.938871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:26.939869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:26.939869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:26.939869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:26.939869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:26.939869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 330/665] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:26.940630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe103b-c2c0-4cd1-aa1d-ee3b2efa6e9b None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:26.940907 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe103b-c2c0-4cd1-aa1d-ee3b2efa6e9b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:26.941179 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe103b-c2c0-4cd1-aa1d-ee3b2efa6e9b None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:26.946645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 331/666] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:26.946645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe103b-c2c0-4cd1-aa1d-ee3b2efa6e9b None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:26.985683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f713438-0280-44a7-b2aa-ed6b7ec87a0c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:27.018040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e941ef-af2d-4113-a285-1926b5d5ec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:27.019135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e941ef-af2d-4113-a285-1926b5d5ec0c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.019560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e941ef-af2d-4113-a285-1926b5d5ec0c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.020002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e941ef-af2d-4113-a285-1926b5d5ec0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.031853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e941ef-af2d-4113-a285-1926b5d5ec0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:27.034225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e941ef-af2d-4113-a285-1926b5d5ec0c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:27.041762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe103b-c2c0-4cd1-aa1d-ee3b2efa6e9b None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-40e941ef-af2d-4113-a285-1926b5d5ec0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:27.055411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:27.056343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:27.057245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:27.057245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:27.057245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:27.057245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:27.057245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 336/667] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:27.058324 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e941ef-af2d-4113-a285-1926b5d5ec0c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:27.081714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b243b-6afe-41cd-9c88-f4306da5ebd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:27.082382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b243b-6afe-41cd-9c88-f4306da5ebd4 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.082666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b243b-6afe-41cd-9c88-f4306da5ebd4 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.083096 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b243b-6afe-41cd-9c88-f4306da5ebd4 None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.083888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b243b-6afe-41cd-9c88-f4306da5ebd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:27.084183 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b243b-6afe-41cd-9c88-f4306da5ebd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:27.085702 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b243b-6afe-41cd-9c88-f4306da5ebd4 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:27.102020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 332/668] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:27.102020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128017cf-baa2-418f-9a87-5afd538da4c9 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.102020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128017cf-baa2-418f-9a87-5afd538da4c9 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.102020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128017cf-baa2-418f-9a87-5afd538da4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.117097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-128017cf-baa2-418f-9a87-5afd538da4c9 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:27.118269 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-128017cf-baa2-418f-9a87-5afd538da4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:27.162419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e941ef-af2d-4113-a285-1926b5d5ec0c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:27.187446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 333/669] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:27.188578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128017cf-baa2-418f-9a87-5afd538da4c9 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:27.217895 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa4a18-69e6-4c36-9331-f24afb09c794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:27.218876 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa4a18-69e6-4c36-9331-f24afb09c794 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.219166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa4a18-69e6-4c36-9331-f24afb09c794 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.219437 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa4a18-69e6-4c36-9331-f24afb09c794 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.220349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa4a18-69e6-4c36-9331-f24afb09c794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:27.220655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa4a18-69e6-4c36-9331-f24afb09c794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:27.222008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa4a18-69e6-4c36-9331-f24afb09c794 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:27.233904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 337/670] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:27.234729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa4a18-69e6-4c36-9331-f24afb09c794 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:27.288470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128017cf-baa2-418f-9a87-5afd538da4c9 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:27.321847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade833b-1939-42b7-a0df-dc1cd66da5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:27.322507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade833b-1939-42b7-a0df-dc1cd66da5d8 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.322795 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade833b-1939-42b7-a0df-dc1cd66da5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.323057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade833b-1939-42b7-a0df-dc1cd66da5d8 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.323774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade833b-1939-42b7-a0df-dc1cd66da5d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:27.324099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade833b-1939-42b7-a0df-dc1cd66da5d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:27.325459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade833b-1939-42b7-a0df-dc1cd66da5d8 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:27.335826 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa4a18-69e6-4c36-9331-f24afb09c794 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:27.337088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 334/671] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:27.338141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade833b-1939-42b7-a0df-dc1cd66da5d8 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:27.368904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965fbb4-f23d-4409-a53d-7f2f5081593a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:27.369563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965fbb4-f23d-4409-a53d-7f2f5081593a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.369846 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965fbb4-f23d-4409-a53d-7f2f5081593a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.370124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965fbb4-f23d-4409-a53d-7f2f5081593a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.370953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965fbb4-f23d-4409-a53d-7f2f5081593a None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=295bb46627e043989e6dba2805607c99) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:27.371266 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965fbb4-f23d-4409-a53d-7f2f5081593a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:27.372758 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965fbb4-f23d-4409-a53d-7f2f5081593a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:27.378670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 338/672] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:27 2022] GET /identity/v3/role_assignments?user.id=295bb46627e043989e6dba2805607c99&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:27.379238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965fbb4-f23d-4409-a53d-7f2f5081593a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:27.437622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade833b-1939-42b7-a0df-dc1cd66da5d8 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:27.474749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adabb2-5533-4ce9-87c6-40ef75c12fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:27.475564 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adabb2-5533-4ce9-87c6-40ef75c12fc9 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.475854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adabb2-5533-4ce9-87c6-40ef75c12fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.476152 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adabb2-5533-4ce9-87c6-40ef75c12fc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.485027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965fbb4-f23d-4409-a53d-7f2f5081593a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:27.487475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adabb2-5533-4ce9-87c6-40ef75c12fc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-f0adabb2-5533-4ce9-87c6-40ef75c12fc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:27.495310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:27.496390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:27.497235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:27.497235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:27.497235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:27.497235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:27.497235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:27.497235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:27.497235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:27.497235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:27.497235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:27.497235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:27.497235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:27.497235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:27.497235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 335/673] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:13:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:27.498504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81883d12-07ab-4821-8367-3284807c38c8 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.498504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81883d12-07ab-4821-8367-3284807c38c8 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.498627 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81883d12-07ab-4821-8367-3284807c38c8 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.500089 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 336/674] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:27.500650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81883d12-07ab-4821-8367-3284807c38c8 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:27.527860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10079eaf-b645-4ada-9d58-946f17a5c360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:27.528712 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10079eaf-b645-4ada-9d58-946f17a5c360 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.529552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10079eaf-b645-4ada-9d58-946f17a5c360 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.529830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10079eaf-b645-4ada-9d58-946f17a5c360 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.537850 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10079eaf-b645-4ada-9d58-946f17a5c360 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:27.539370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10079eaf-b645-4ada-9d58-946f17a5c360 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-10079eaf-b645-4ada-9d58-946f17a5c360 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:27.554603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:27.555599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:27.557553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:27.557553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:27.557553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:27.557553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:27.557553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:27.557553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:27.557553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:27.557553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:27.560511 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 339/675] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:27.561507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10079eaf-b645-4ada-9d58-946f17a5c360 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:27.616334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81883d12-07ab-4821-8367-3284807c38c8 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:27.653826 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e94b2-abf8-433a-921d-b66b4ba11d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:27.654954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e94b2-abf8-433a-921d-b66b4ba11d5c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.655257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e94b2-abf8-433a-921d-b66b4ba11d5c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.655519 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e94b2-abf8-433a-921d-b66b4ba11d5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.662302 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e94b2-abf8-433a-921d-b66b4ba11d5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:27.663738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e94b2-abf8-433a-921d-b66b4ba11d5c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:27.676961 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10079eaf-b645-4ada-9d58-946f17a5c360 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-646e94b2-abf8-433a-921d-b66b4ba11d5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:27.678258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:27.679127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:27.680636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:27.680636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:27.680636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:27.680636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:27.680636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:27.680636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:27.680636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:27.680636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:27.680636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 337/676] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:27.681141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c37c5-6be4-431e-850a-0caad50518ae None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.681355 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c37c5-6be4-431e-850a-0caad50518ae None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.681613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c37c5-6be4-431e-850a-0caad50518ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.698163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-cd5c37c5-6be4-431e-850a-0caad50518ae None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:27.699598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cd5c37c5-6be4-431e-850a-0caad50518ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:27.712648 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf17f4b-4a47-41d5-9685-bc97c2c6db26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:27.713312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf17f4b-4a47-41d5-9685-bc97c2c6db26 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.713596 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf17f4b-4a47-41d5-9685-bc97c2c6db26 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.714164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf17f4b-4a47-41d5-9685-bc97c2c6db26 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.714609 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf17f4b-4a47-41d5-9685-bc97c2c6db26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:27.714933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf17f4b-4a47-41d5-9685-bc97c2c6db26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:27.716932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf17f4b-4a47-41d5-9685-bc97c2c6db26 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:27.725088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 340/677] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:27.725884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf17f4b-4a47-41d5-9685-bc97c2c6db26 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:27.780119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 338/678] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:27.780119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c37c5-6be4-431e-850a-0caad50518ae None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:27.837731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf17f4b-4a47-41d5-9685-bc97c2c6db26 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:27.867321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c37c5-6be4-431e-850a-0caad50518ae None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:27.870983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8ddbd-a816-402d-8336-7a18761b42b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:27.871644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8ddbd-a816-402d-8336-7a18761b42b3 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.871947 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8ddbd-a816-402d-8336-7a18761b42b3 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.872224 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8ddbd-a816-402d-8336-7a18761b42b3 None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.873760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8ddbd-a816-402d-8336-7a18761b42b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:27.874069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8ddbd-a816-402d-8336-7a18761b42b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:27.875481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8ddbd-a816-402d-8336-7a18761b42b3 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:27.887074 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 341/679] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:27.888368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8ddbd-a816-402d-8336-7a18761b42b3 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:27.900440 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e25863-d842-4eee-9314-9f920592a8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:27.901100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e25863-d842-4eee-9314-9f920592a8c0 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:27.901376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e25863-d842-4eee-9314-9f920592a8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:27.901644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e25863-d842-4eee-9314-9f920592a8c0 None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:27.902430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e25863-d842-4eee-9314-9f920592a8c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:27.902723 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e25863-d842-4eee-9314-9f920592a8c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:27.904990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e25863-d842-4eee-9314-9f920592a8c0 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:27.916547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 339/680] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:27.917445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e25863-d842-4eee-9314-9f920592a8c0 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:27.988478 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8ddbd-a816-402d-8336-7a18761b42b3 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:28.012407 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e25863-d842-4eee-9314-9f920592a8c0 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:28.020758 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a7ac2-8a18-4f38-9514-dbd17f63026e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:28.021550 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a7ac2-8a18-4f38-9514-dbd17f63026e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:28.021828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a7ac2-8a18-4f38-9514-dbd17f63026e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:28.022098 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a7ac2-8a18-4f38-9514-dbd17f63026e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:28.032680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a7ac2-8a18-4f38-9514-dbd17f63026e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:28.034114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a7ac2-8a18-4f38-9514-dbd17f63026e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:28.045270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ec1d9-b5e6-4cf2-b96a-74be70f3bc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:28.046210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ec1d9-b5e6-4cf2-b96a-74be70f3bc53 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:28.046494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ec1d9-b5e6-4cf2-b96a-74be70f3bc53 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:28.046767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ec1d9-b5e6-4cf2-b96a-74be70f3bc53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-025a7ac2-8a18-4f38-9514-dbd17f63026e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:28.051733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:28.052734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:28.053662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:28.053662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:28.053662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:28.053662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:28.053662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 342/681] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:28.053935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a7ac2-8a18-4f38-9514-dbd17f63026e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:28.062456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ec1d9-b5e6-4cf2-b96a-74be70f3bc53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-b80ec1d9-b5e6-4cf2-b96a-74be70f3bc53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:28.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:28.071203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:28.072064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:28.072064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:28.072064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:28.072064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:28.072064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:28.072064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:28.072064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:28.072064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:28.072064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:28.072064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:28.072064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:28.072064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:28.072064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 340/682] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:28.072753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ec1d9-b5e6-4cf2-b96a-74be70f3bc53 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:28.151883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a7ac2-8a18-4f38-9514-dbd17f63026e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:28.174081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ec1d9-b5e6-4cf2-b96a-74be70f3bc53 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:28.185977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369f5e9-3579-4e16-8716-effc966b4e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:28.186636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369f5e9-3579-4e16-8716-effc966b4e04 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:28.186919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369f5e9-3579-4e16-8716-effc966b4e04 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:28.187193 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369f5e9-3579-4e16-8716-effc966b4e04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:28.188052 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369f5e9-3579-4e16-8716-effc966b4e04 None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=72caf90f411f495bb22aa85df4d25eb3) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:28.188366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369f5e9-3579-4e16-8716-effc966b4e04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:28.189795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369f5e9-3579-4e16-8716-effc966b4e04 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:28.204213 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 343/683] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:28 2022] GET /identity/v3/role_assignments?user.id=72caf90f411f495bb22aa85df4d25eb3&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:28.205689 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369f5e9-3579-4e16-8716-effc966b4e04 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:28.207856 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d25b10-2922-4bb1-b4a6-810005746b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:28.208680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d25b10-2922-4bb1-b4a6-810005746b17 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:28.209060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d25b10-2922-4bb1-b4a6-810005746b17 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:28.209450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d25b10-2922-4bb1-b4a6-810005746b17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:28.210419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d25b10-2922-4bb1-b4a6-810005746b17 None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=0d852975a2e84925a45775a16cc52f32) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:28.210833 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d25b10-2922-4bb1-b4a6-810005746b17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:28.212472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d25b10-2922-4bb1-b4a6-810005746b17 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:28.228400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 341/684] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:28 2022] GET /identity/v3/role_assignments?user.id=0d852975a2e84925a45775a16cc52f32&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:28.229582 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d25b10-2922-4bb1-b4a6-810005746b17 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:28.334828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369f5e9-3579-4e16-8716-effc966b4e04 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:28.377470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059d118-dee8-423d-96f6-06367d08b917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:28.379579 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059d118-dee8-423d-96f6-06367d08b917 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:28.380248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059d118-dee8-423d-96f6-06367d08b917 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:28.380735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059d118-dee8-423d-96f6-06367d08b917 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:28.381651 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059d118-dee8-423d-96f6-06367d08b917 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:28.382141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059d118-dee8-423d-96f6-06367d08b917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:28.383740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059d118-dee8-423d-96f6-06367d08b917 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:28.403581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 344/685] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:28.406087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059d118-dee8-423d-96f6-06367d08b917 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:28.445660 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d25b10-2922-4bb1-b4a6-810005746b17 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:28.483809 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02e333-29f2-46b0-8c85-3d29f7041c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:28.484844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02e333-29f2-46b0-8c85-3d29f7041c81 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:28.485323 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02e333-29f2-46b0-8c85-3d29f7041c81 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:28.485791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02e333-29f2-46b0-8c85-3d29f7041c81 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:28.486741 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02e333-29f2-46b0-8c85-3d29f7041c81 None admin] RBAC: Adding query filter params (name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:28.487222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02e333-29f2-46b0-8c85-3d29f7041c81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:28.488823 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02e333-29f2-46b0-8c85-3d29f7041c81 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:28.498981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 342/686] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:28.506896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02e333-29f2-46b0-8c85-3d29f7041c81 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:28.545230 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059d118-dee8-423d-96f6-06367d08b917 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:28.590317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62156eb2-600c-4da2-9a8f-370a1f7f0e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:28.592236 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62156eb2-600c-4da2-9a8f-370a1f7f0e38 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:28.593239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62156eb2-600c-4da2-9a8f-370a1f7f0e38 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:28.593754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62156eb2-600c-4da2-9a8f-370a1f7f0e38 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:28.604381 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62156eb2-600c-4da2-9a8f-370a1f7f0e38 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:28.606158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62156eb2-600c-4da2-9a8f-370a1f7f0e38 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-62156eb2-600c-4da2-9a8f-370a1f7f0e38 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:28.627487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:28.628541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:28.629477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:28.629477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:28.629477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:28.629477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:28.629477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:28.629477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:28.629477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:28.629477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:28.629477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 04:13:28.629477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:28.632456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 345/687] 10.222.0.44 () {56 vars in 1093 bytes} [Mon May 9 04:13:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:28.637818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02e333-29f2-46b0-8c85-3d29f7041c81 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:28.640165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62156eb2-600c-4da2-9a8f-370a1f7f0e38 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:28.673066 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaa023-f7ad-4b32-b655-bdf7145edf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:28.674201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaa023-f7ad-4b32-b655-bdf7145edf20 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:28.674699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaa023-f7ad-4b32-b655-bdf7145edf20 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:28.675165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaa023-f7ad-4b32-b655-bdf7145edf20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:28.687244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaa023-f7ad-4b32-b655-bdf7145edf20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:28.689171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaa023-f7ad-4b32-b655-bdf7145edf20 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-5afaa023-f7ad-4b32-b655-bdf7145edf20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:28.707484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:28.708444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:28.709400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:28.709400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:28.709400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:28.709400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:28.711572 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 343/688] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:28.717680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaa023-f7ad-4b32-b655-bdf7145edf20 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:28.755877 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62156eb2-600c-4da2-9a8f-370a1f7f0e38 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:28.792869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4f822-5700-4d20-9d7f-87dd8129a146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:28.793915 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4f822-5700-4d20-9d7f-87dd8129a146 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:28.794428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4f822-5700-4d20-9d7f-87dd8129a146 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:28.795432 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4f822-5700-4d20-9d7f-87dd8129a146 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:28.796519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4f822-5700-4d20-9d7f-87dd8129a146 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:28.797029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4f822-5700-4d20-9d7f-87dd8129a146 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:28.798651 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4f822-5700-4d20-9d7f-87dd8129a146 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:28.817513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 346/689] 10.222.0.44 () {56 vars in 1133 bytes} [Mon May 9 04:13:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:28.822498 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaa023-f7ad-4b32-b655-bdf7145edf20 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:28.823428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4f822-5700-4d20-9d7f-87dd8129a146 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:28.867016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b41e5-a55f-414d-a901-03f2a72c00f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:28.867689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b41e5-a55f-414d-a901-03f2a72c00f6 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:28.867999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b41e5-a55f-414d-a901-03f2a72c00f6 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:28.868281 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b41e5-a55f-414d-a901-03f2a72c00f6 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:28.869024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b41e5-a55f-414d-a901-03f2a72c00f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:28.869315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b41e5-a55f-414d-a901-03f2a72c00f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:28.870735 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b41e5-a55f-414d-a901-03f2a72c00f6 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:28.898318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 344/690] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:28.903996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b41e5-a55f-414d-a901-03f2a72c00f6 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:28.926100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4f822-5700-4d20-9d7f-87dd8129a146 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:28.960726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b23048-ded5-44b5-9fd4-178a959df251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:28.961744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b23048-ded5-44b5-9fd4-178a959df251 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:28.962226 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b23048-ded5-44b5-9fd4-178a959df251 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:28.962720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b23048-ded5-44b5-9fd4-178a959df251 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:29.030733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b23048-ded5-44b5-9fd4-178a959df251 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:29.032554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b23048-ded5-44b5-9fd4-178a959df251 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:29.092064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b41e5-a55f-414d-a901-03f2a72c00f6 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:29.131049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 347/691] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 04:13:29.136672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b23048-ded5-44b5-9fd4-178a959df251 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:29.186062 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3dd8a-49ca-4b82-8285-714281ae7f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:29.186789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3dd8a-49ca-4b82-8285-714281ae7f84 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:29.187071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3dd8a-49ca-4b82-8285-714281ae7f84 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:29.187343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3dd8a-49ca-4b82-8285-714281ae7f84 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:29.204698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3dd8a-49ca-4b82-8285-714281ae7f84 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:29.206622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3dd8a-49ca-4b82-8285-714281ae7f84 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-67d3dd8a-49ca-4b82-8285-714281ae7f84 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:29.229069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:29.230044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:29.230960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:29.230960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:29.230960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:29.230960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:29.230960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:29.230960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:29.230960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:29.230960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:29.230960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 04:13:29.230960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:29.236520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 345/692] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:13:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:29.240434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3dd8a-49ca-4b82-8285-714281ae7f84 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:29.245208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b23048-ded5-44b5-9fd4-178a959df251 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:29.312832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6ef13-ce6c-4ff4-9758-3f1e97635cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:29.313982 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6ef13-ce6c-4ff4-9758-3f1e97635cf9 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:29.314498 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6ef13-ce6c-4ff4-9758-3f1e97635cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:29.315008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6ef13-ce6c-4ff4-9758-3f1e97635cf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:29.328028 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6ef13-ce6c-4ff4-9758-3f1e97635cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:29.330053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6ef13-ce6c-4ff4-9758-3f1e97635cf9 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-16c6ef13-ce6c-4ff4-9758-3f1e97635cf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:29.348507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:29.349445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:29.350335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:29.350335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:29.350335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:29.350335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:29.352617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 348/693] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:29.358624 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6ef13-ce6c-4ff4-9758-3f1e97635cf9 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:29.428409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3dd8a-49ca-4b82-8285-714281ae7f84 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:29.457437 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6ef13-ce6c-4ff4-9758-3f1e97635cf9 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:29.471056 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93232c23-69ad-4fe1-a903-e2d2f06ba3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:29.471734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93232c23-69ad-4fe1-a903-e2d2f06ba3c5 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:29.472047 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93232c23-69ad-4fe1-a903-e2d2f06ba3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:29.472325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93232c23-69ad-4fe1-a903-e2d2f06ba3c5 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:29.473204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93232c23-69ad-4fe1-a903-e2d2f06ba3c5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:29.473476 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93232c23-69ad-4fe1-a903-e2d2f06ba3c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:29.475025 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93232c23-69ad-4fe1-a903-e2d2f06ba3c5 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:29.494042 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 346/694] 10.222.0.44 () {56 vars in 1135 bytes} [Mon May 9 04:13:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:29.495031 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc6d10-1b9c-4f0a-aee2-39b65cc6fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:29.495960 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc6d10-1b9c-4f0a-aee2-39b65cc6fb65 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:29.496459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc6d10-1b9c-4f0a-aee2-39b65cc6fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:29.496943 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc6d10-1b9c-4f0a-aee2-39b65cc6fb65 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:29.497902 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc6d10-1b9c-4f0a-aee2-39b65cc6fb65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:29.498703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc6d10-1b9c-4f0a-aee2-39b65cc6fb65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:29.499496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93232c23-69ad-4fe1-a903-e2d2f06ba3c5 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:29.500500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc6d10-1b9c-4f0a-aee2-39b65cc6fb65 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:29.512670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 349/695] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:29.518312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc6d10-1b9c-4f0a-aee2-39b65cc6fb65 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:29.600903 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93232c23-69ad-4fe1-a903-e2d2f06ba3c5 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:29.617513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc6d10-1b9c-4f0a-aee2-39b65cc6fb65 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:29.635452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a841ac-7874-43a1-a01a-e5e73b7e3846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:29.636134 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a841ac-7874-43a1-a01a-e5e73b7e3846 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:29.636419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a841ac-7874-43a1-a01a-e5e73b7e3846 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:29.636694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a841ac-7874-43a1-a01a-e5e73b7e3846 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:29.652103 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6daa3-5cca-4118-b8bc-6c8f36e93235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:29.653237 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6daa3-5cca-4118-b8bc-6c8f36e93235 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:29.653746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6daa3-5cca-4118-b8bc-6c8f36e93235 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:29.654222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6daa3-5cca-4118-b8bc-6c8f36e93235 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:29.661181 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6daa3-5cca-4118-b8bc-6c8f36e93235 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:29.662836 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6daa3-5cca-4118-b8bc-6c8f36e93235 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-06e6daa3-5cca-4118-b8bc-6c8f36e93235 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:29.677594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:29.678514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:29.679415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:29.679415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:29.679415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:29.679415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:29.679415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:29.679415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:29.679415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:29.679415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:29.681889 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 350/696] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:29.688288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6daa3-5cca-4118-b8bc-6c8f36e93235 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:29.706611 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a841ac-7874-43a1-a01a-e5e73b7e3846 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:29.708155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a841ac-7874-43a1-a01a-e5e73b7e3846 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:29.789920 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6daa3-5cca-4118-b8bc-6c8f36e93235 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:29.818616 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 347/697] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 04:13:29.824914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a841ac-7874-43a1-a01a-e5e73b7e3846 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:29.826773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1de481-0c5c-42f1-893c-268e86c8adac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:29.827714 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1de481-0c5c-42f1-893c-268e86c8adac None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:29.828266 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1de481-0c5c-42f1-893c-268e86c8adac None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:29.828998 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1de481-0c5c-42f1-893c-268e86c8adac None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:29.830029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1de481-0c5c-42f1-893c-268e86c8adac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:29.830546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1de481-0c5c-42f1-893c-268e86c8adac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:29.832275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1de481-0c5c-42f1-893c-268e86c8adac None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:29.845012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 351/698] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:29.851277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1de481-0c5c-42f1-893c-268e86c8adac None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:29.926838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a841ac-7874-43a1-a01a-e5e73b7e3846 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:29.964412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482d837-6d12-4ed2-887f-581c2800517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:29.965140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482d837-6d12-4ed2-887f-581c2800517c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:29.965422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482d837-6d12-4ed2-887f-581c2800517c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:29.965699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482d837-6d12-4ed2-887f-581c2800517c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:29.977625 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482d837-6d12-4ed2-887f-581c2800517c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:29.979120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482d837-6d12-4ed2-887f-581c2800517c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-4482d837-6d12-4ed2-887f-581c2800517c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:30.007347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:30.008354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:30.009260 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:30.009260 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:30.009260 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:30.009260 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:30.009260 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 348/699] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:30.015086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482d837-6d12-4ed2-887f-581c2800517c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:30.079605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1de481-0c5c-42f1-893c-268e86c8adac None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:30.114487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f604d4c-87d7-4ea7-8740-7db0e68f3068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:30.115508 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f604d4c-87d7-4ea7-8740-7db0e68f3068 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:30.116189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f604d4c-87d7-4ea7-8740-7db0e68f3068 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:30.116666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f604d4c-87d7-4ea7-8740-7db0e68f3068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:30.117810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f604d4c-87d7-4ea7-8740-7db0e68f3068 None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=faf6107735424ff7901614953e7df403) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:30.118331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f604d4c-87d7-4ea7-8740-7db0e68f3068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:30.119982 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f604d4c-87d7-4ea7-8740-7db0e68f3068 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:30.126074 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 352/700] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:29 2022] GET /identity/v3/role_assignments?user.id=faf6107735424ff7901614953e7df403&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=e59113c0366049e889df3f55eed90c72 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:30.188645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482d837-6d12-4ed2-887f-581c2800517c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:30.231293 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590ee4e-ddbf-43e1-a1db-06f7f3bfe665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:30.232672 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590ee4e-ddbf-43e1-a1db-06f7f3bfe665 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:30.233010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590ee4e-ddbf-43e1-a1db-06f7f3bfe665 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:30.233292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590ee4e-ddbf-43e1-a1db-06f7f3bfe665 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:30.234031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590ee4e-ddbf-43e1-a1db-06f7f3bfe665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:30.234334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590ee4e-ddbf-43e1-a1db-06f7f3bfe665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:30.235717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590ee4e-ddbf-43e1-a1db-06f7f3bfe665 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:30.248591 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 349/701] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:30.254621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f604d4c-87d7-4ea7-8740-7db0e68f3068 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:30.281115 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eef74-3abf-4956-826f-f6efb2b029a2 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:30.281780 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eef74-3abf-4956-826f-f6efb2b029a2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:30.282285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eef74-3abf-4956-826f-f6efb2b029a2 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:30.284092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 350/702] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:13:30.294656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b22ff1-7849-4f4c-ba3e-c3c66ede8d18 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:30.297156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b22ff1-7849-4f4c-ba3e-c3c66ede8d18 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:30.297680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b22ff1-7849-4f4c-ba3e-c3c66ede8d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:30.337460 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-72b22ff1-7849-4f4c-ba3e-c3c66ede8d18 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:30.340176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-72b22ff1-7849-4f4c-ba3e-c3c66ede8d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:30.443720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f604d4c-87d7-4ea7-8740-7db0e68f3068 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:30.453919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 351/703] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:30.479741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd36ddb-5a3f-47b8-9eb0-068f1a2bf0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:30.480568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd36ddb-5a3f-47b8-9eb0-068f1a2bf0f3 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:30.480851 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd36ddb-5a3f-47b8-9eb0-068f1a2bf0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:30.481121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd36ddb-5a3f-47b8-9eb0-068f1a2bf0f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:30.487754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd36ddb-5a3f-47b8-9eb0-068f1a2bf0f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:30.489157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd36ddb-5a3f-47b8-9eb0-068f1a2bf0f3 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:30.499699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b22ff1-7849-4f4c-ba3e-c3c66ede8d18 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-4cd36ddb-5a3f-47b8-9eb0-068f1a2bf0f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:30.515011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:30.515953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:30.516835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:30.516835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:30.516835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:30.516835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:30.516835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:30.516835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:30.516835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:30.516835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:30.516835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 353/704] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:30.525163 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd36ddb-5a3f-47b8-9eb0-068f1a2bf0f3 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:30.644155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b22ff1-7849-4f4c-ba3e-c3c66ede8d18 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:30.678334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb971f4-0f8d-47e9-b14a-d647222192b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:30.679131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb971f4-0f8d-47e9-b14a-d647222192b1 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:30.679411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb971f4-0f8d-47e9-b14a-d647222192b1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:30.679685 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb971f4-0f8d-47e9-b14a-d647222192b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:30.690862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb971f4-0f8d-47e9-b14a-d647222192b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:30.692363 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb971f4-0f8d-47e9-b14a-d647222192b1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-ecb971f4-0f8d-47e9-b14a-d647222192b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:30.710505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:30.712834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:30.714870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:30.714870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:30.714870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:30.714870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:30.714870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 352/705] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:30.718591 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb971f4-0f8d-47e9-b14a-d647222192b1 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:30.745758 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd36ddb-5a3f-47b8-9eb0-068f1a2bf0f3 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:30.814515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aafe2-0ff6-4eb1-8e0e-dd55b7875ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:30.815203 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aafe2-0ff6-4eb1-8e0e-dd55b7875ea6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:30.815485 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aafe2-0ff6-4eb1-8e0e-dd55b7875ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:30.815761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aafe2-0ff6-4eb1-8e0e-dd55b7875ea6 None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:30.816896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aafe2-0ff6-4eb1-8e0e-dd55b7875ea6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:30.816896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aafe2-0ff6-4eb1-8e0e-dd55b7875ea6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:30.822939 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aafe2-0ff6-4eb1-8e0e-dd55b7875ea6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:30.831441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 354/706] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:30.837454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aafe2-0ff6-4eb1-8e0e-dd55b7875ea6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:30.908558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb971f4-0f8d-47e9-b14a-d647222192b1 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:30.943053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd48c5-dc65-41f1-b088-ffb071f69c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:30.943775 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd48c5-dc65-41f1-b088-ffb071f69c13 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:30.944103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd48c5-dc65-41f1-b088-ffb071f69c13 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:30.944381 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd48c5-dc65-41f1-b088-ffb071f69c13 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:30.945111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd48c5-dc65-41f1-b088-ffb071f69c13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:30.945417 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd48c5-dc65-41f1-b088-ffb071f69c13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:30.946797 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd48c5-dc65-41f1-b088-ffb071f69c13 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:30.959172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 353/707] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:30.967152 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd48c5-dc65-41f1-b088-ffb071f69c13 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:31.005545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aafe2-0ff6-4eb1-8e0e-dd55b7875ea6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:31.062889 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef1839-100a-4f31-baa4-d96553dd512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:31.063632 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef1839-100a-4f31-baa4-d96553dd512f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:31.063933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef1839-100a-4f31-baa4-d96553dd512f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:31.064213 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef1839-100a-4f31-baa4-d96553dd512f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:31.065250 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef1839-100a-4f31-baa4-d96553dd512f None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=c46e7cc6407f44f08a659288e3ad0767) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:31.065567 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef1839-100a-4f31-baa4-d96553dd512f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:31.068739 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef1839-100a-4f31-baa4-d96553dd512f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:31.079175 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 355/708] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:30 2022] GET /identity/v3/role_assignments?user.id=c46e7cc6407f44f08a659288e3ad0767&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:31.081616 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b8402-00ad-4bc9-ba3e-f66922e4e3a4 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:31.081908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b8402-00ad-4bc9-ba3e-f66922e4e3a4 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:31.082186 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b8402-00ad-4bc9-ba3e-f66922e4e3a4 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:31.095938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 356/709] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:31.095938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859e2ec-f0df-4016-96bb-c96e890beded None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:31.095938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859e2ec-f0df-4016-96bb-c96e890beded None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:31.095938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859e2ec-f0df-4016-96bb-c96e890beded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:31.110344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-1859e2ec-f0df-4016-96bb-c96e890beded None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:31.111855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1859e2ec-f0df-4016-96bb-c96e890beded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:31.155651 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd48c5-dc65-41f1-b088-ffb071f69c13 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:31.233091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9963bd7-3936-4bb4-a118-bc149ceec3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:31.234159 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9963bd7-3936-4bb4-a118-bc149ceec3d9 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:31.234617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9963bd7-3936-4bb4-a118-bc149ceec3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:31.235078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9963bd7-3936-4bb4-a118-bc149ceec3d9 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:31.236018 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9963bd7-3936-4bb4-a118-bc149ceec3d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:31.242250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9963bd7-3936-4bb4-a118-bc149ceec3d9 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:31.249872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 357/710] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:31.251640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da93fd-489f-4120-8a10-d82fd01aec94 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:31.252179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da93fd-489f-4120-8a10-d82fd01aec94 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:31.252726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da93fd-489f-4120-8a10-d82fd01aec94 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:31.256868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 358/711] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:31.261374 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf38e4-d0ff-4444-9fd7-405baa9c3628 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:31.261945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf38e4-d0ff-4444-9fd7-405baa9c3628 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:31.262519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf38e4-d0ff-4444-9fd7-405baa9c3628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:31.275281 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 354/712] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:13:30 2022] POST /identity/v3/users => generated 254 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:13:31.276243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9963bd7-3936-4bb4-a118-bc149ceec3d9 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:31.288500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-eecf38e4-d0ff-4444-9fd7-405baa9c3628 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:31.290835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-eecf38e4-d0ff-4444-9fd7-405baa9c3628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:31.374205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 359/713] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:31.407028 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9963bd7-3936-4bb4-a118-bc149ceec3d9 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:31.412461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf38e4-d0ff-4444-9fd7-405baa9c3628 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:31.451138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23289a2-11ae-4469-965c-ad0a1d15ca17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:31.452188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23289a2-11ae-4469-965c-ad0a1d15ca17 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:31.452491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23289a2-11ae-4469-965c-ad0a1d15ca17 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:31.452765 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23289a2-11ae-4469-965c-ad0a1d15ca17 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:31.453870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23289a2-11ae-4469-965c-ad0a1d15ca17 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:31.455714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23289a2-11ae-4469-965c-ad0a1d15ca17 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-e23289a2-11ae-4469-965c-ad0a1d15ca17 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:31.474680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:31.475704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 04:13:31.476586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:31.477366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 355/714] 10.222.0.44 () {56 vars in 1129 bytes} [Mon May 9 04:13:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:31.485409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23289a2-11ae-4469-965c-ad0a1d15ca17 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:31.542280 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf38e4-d0ff-4444-9fd7-405baa9c3628 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:31.579630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130def0-d064-4c17-b9cb-1c57131e9e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:31.580748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130def0-d064-4c17-b9cb-1c57131e9e8d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:31.581209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130def0-d064-4c17-b9cb-1c57131e9e8d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:31.581675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130def0-d064-4c17-b9cb-1c57131e9e8d None admin] PATH_INFO: `/v3/services/network` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:31.582507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130def0-d064-4c17-b9cb-1c57131e9e8d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:31.583857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130def0-d064-4c17-b9cb-1c57131e9e8d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-9130def0-d064-4c17-b9cb-1c57131e9e8d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:31.597190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:31.598215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:31.599053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 360/715] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:31 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:31.604448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130def0-d064-4c17-b9cb-1c57131e9e8d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:31.623338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23289a2-11ae-4469-965c-ad0a1d15ca17 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:31.668373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd50b4-44f8-4198-a5ba-fd61f4ec2bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:31.670070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd50b4-44f8-4198-a5ba-fd61f4ec2bdf None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:31.670555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd50b4-44f8-4198-a5ba-fd61f4ec2bdf None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:31.671311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd50b4-44f8-4198-a5ba-fd61f4ec2bdf None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:31.672501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd50b4-44f8-4198-a5ba-fd61f4ec2bdf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:31.673229 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd50b4-44f8-4198-a5ba-fd61f4ec2bdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:31.675317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd50b4-44f8-4198-a5ba-fd61f4ec2bdf None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:31.681280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 356/716] 10.222.0.44 () {56 vars in 1124 bytes} [Mon May 9 04:13:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:31.686677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd50b4-44f8-4198-a5ba-fd61f4ec2bdf None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:31.723078 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130def0-d064-4c17-b9cb-1c57131e9e8d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:31.757541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f3a4e-5088-4f2d-a361-693a22752437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:31.758423 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f3a4e-5088-4f2d-a361-693a22752437 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:31.758544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f3a4e-5088-4f2d-a361-693a22752437 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:31.758777 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f3a4e-5088-4f2d-a361-693a22752437 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:31.759483 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f3a4e-5088-4f2d-a361-693a22752437 None admin] RBAC: Adding query filter params (name=network) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:31.759786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f3a4e-5088-4f2d-a361-693a22752437 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:31.761390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f3a4e-5088-4f2d-a361-693a22752437 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:31.767313 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 361/717] 10.222.0.44 () {56 vars in 1112 bytes} [Mon May 9 04:13:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:31.773362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f3a4e-5088-4f2d-a361-693a22752437 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:31.796854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd50b4-44f8-4198-a5ba-fd61f4ec2bdf None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:31.830664 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3aaead-9110-4a7b-82be-cc70aaaf666a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:31.831664 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aaead-9110-4a7b-82be-cc70aaaf666a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:31.832180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aaead-9110-4a7b-82be-cc70aaaf666a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:31.832681 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aaead-9110-4a7b-82be-cc70aaaf666a None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:31.833577 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3aaead-9110-4a7b-82be-cc70aaaf666a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:31.834075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3aaead-9110-4a7b-82be-cc70aaaf666a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:31.835739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3aaead-9110-4a7b-82be-cc70aaaf666a None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:31.842239 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 357/718] 10.222.0.44 () {56 vars in 1124 bytes} [Mon May 9 04:13:31 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:31.877576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f3a4e-5088-4f2d-a361-693a22752437 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:31.914692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670b117-7542-4b68-b560-90f7e0294e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:31.915379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670b117-7542-4b68-b560-90f7e0294e5e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:31.915660 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670b117-7542-4b68-b560-90f7e0294e5e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:31.915956 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670b117-7542-4b68-b560-90f7e0294e5e None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:31.916672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670b117-7542-4b68-b560-90f7e0294e5e None admin] RBAC: Adding query filter params (type=network) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:31.916974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670b117-7542-4b68-b560-90f7e0294e5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:31.918333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670b117-7542-4b68-b560-90f7e0294e5e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:31.925570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 362/719] 10.222.0.44 () {56 vars in 1112 bytes} [Mon May 9 04:13:31 2022] GET /identity/v3/services?type=network => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:32.890380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69807648-4d34-448a-b737-648d394ae2d9 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:32.890932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69807648-4d34-448a-b737-648d394ae2d9 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:32.890932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69807648-4d34-448a-b737-648d394ae2d9 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:32.892578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 358/720] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:32.910119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b75cb9-b68f-4b2d-bea2-ca07aded2715 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:32.910590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b75cb9-b68f-4b2d-bea2-ca07aded2715 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:32.910694 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b75cb9-b68f-4b2d-bea2-ca07aded2715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:32.927682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-c1b75cb9-b68f-4b2d-bea2-ca07aded2715 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:32.928880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c1b75cb9-b68f-4b2d-bea2-ca07aded2715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:33.023649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 363/721] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:13:33.049492 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69807648-4d34-448a-b737-648d394ae2d9 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:33.176981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69807648-4d34-448a-b737-648d394ae2d9 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:33.211568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1422c1-73ff-49e6-8d7b-b8989a20254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:33.212415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1422c1-73ff-49e6-8d7b-b8989a20254a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:33.212692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1422c1-73ff-49e6-8d7b-b8989a20254a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:33.212962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1422c1-73ff-49e6-8d7b-b8989a20254a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:33.226787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1422c1-73ff-49e6-8d7b-b8989a20254a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-6c1422c1-73ff-49e6-8d7b-b8989a20254a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:33.235147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:33.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:33.237080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:33.237080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:33.237080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:33.237080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:33.237080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:33.237080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:33.237080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:33.237080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:33.237080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:33.237080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:33.237080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:33.237080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:33.239766 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 359/722] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:13:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:33.245717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b75cb9-b68f-4b2d-bea2-ca07aded2715 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:33.405767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b75cb9-b68f-4b2d-bea2-ca07aded2715 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:33.457215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ed9c2-c38a-4d7e-9abd-4168a7e2e60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:33.457886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ed9c2-c38a-4d7e-9abd-4168a7e2e60c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:33.458166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ed9c2-c38a-4d7e-9abd-4168a7e2e60c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:33.458436 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ed9c2-c38a-4d7e-9abd-4168a7e2e60c None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:33.459179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ed9c2-c38a-4d7e-9abd-4168a7e2e60c None admin] RBAC: Adding query filter params (name=admin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:33.459480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ed9c2-c38a-4d7e-9abd-4168a7e2e60c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:33.460853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ed9c2-c38a-4d7e-9abd-4168a7e2e60c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:33.471294 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 364/723] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:33.476886 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1422c1-73ff-49e6-8d7b-b8989a20254a None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:33.597525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1422c1-73ff-49e6-8d7b-b8989a20254a None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:33.630220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee5ee0-1992-46c4-848c-94e4c002a4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:33.631357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee5ee0-1992-46c4-848c-94e4c002a4e0 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:33.631910 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee5ee0-1992-46c4-848c-94e4c002a4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:33.632400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee5ee0-1992-46c4-848c-94e4c002a4e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:33.643677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee5ee0-1992-46c4-848c-94e4c002a4e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:33.645378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee5ee0-1992-46c4-848c-94e4c002a4e0 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-2fee5ee0-1992-46c4-848c-94e4c002a4e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:33.662792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:33.663712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:33.664631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:33.664631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:33.664631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:33.664631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:33.666809 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 360/724] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:33.672177 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ed9c2-c38a-4d7e-9abd-4168a7e2e60c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:33.773695 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ed9c2-c38a-4d7e-9abd-4168a7e2e60c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:33.807595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5514bda-e142-40aa-a129-bd862805b8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:33.808301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5514bda-e142-40aa-a129-bd862805b8d7 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:33.808588 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5514bda-e142-40aa-a129-bd862805b8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:33.808868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5514bda-e142-40aa-a129-bd862805b8d7 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:33.809592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5514bda-e142-40aa-a129-bd862805b8d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:33.810406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5514bda-e142-40aa-a129-bd862805b8d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:33.812444 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5514bda-e142-40aa-a129-bd862805b8d7 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:33.824634 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 365/725] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:33.830788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee5ee0-1992-46c4-848c-94e4c002a4e0 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:33.940381 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee5ee0-1992-46c4-848c-94e4c002a4e0 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:33.975734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee34ae-17ca-4ae1-8848-7c2c70c3c364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:33.976563 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee34ae-17ca-4ae1-8848-7c2c70c3c364 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:33.976849 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee34ae-17ca-4ae1-8848-7c2c70c3c364 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:33.977125 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee34ae-17ca-4ae1-8848-7c2c70c3c364 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:33.986045 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee34ae-17ca-4ae1-8848-7c2c70c3c364 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:33.987505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee34ae-17ca-4ae1-8848-7c2c70c3c364 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-17ee34ae-17ca-4ae1-8848-7c2c70c3c364 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:34.002158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:34.004568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:34.006887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:34.006887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:34.006887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:34.006887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:34.006887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:34.006887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:34.006887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:34.006887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:34.006887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 04:13:34.006887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:34.006887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 361/726] 10.222.0.44 () {56 vars in 1093 bytes} [Mon May 9 04:13:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:34.008878 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5514bda-e142-40aa-a129-bd862805b8d7 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:34.070462 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853512a2-a581-4d1e-8bc8-847d5b7cd53d None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.070798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853512a2-a581-4d1e-8bc8-847d5b7cd53d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.071014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853512a2-a581-4d1e-8bc8-847d5b7cd53d None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.072592 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 362/727] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:13:34.089879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2e435-304b-44a0-a3eb-cc55e443ec92 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.090504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2e435-304b-44a0-a3eb-cc55e443ec92 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.091027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2e435-304b-44a0-a3eb-cc55e443ec92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.124919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-eba2e435-304b-44a0-a3eb-cc55e443ec92 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:34.126108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-eba2e435-304b-44a0-a3eb-cc55e443ec92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:34.135971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5514bda-e142-40aa-a129-bd862805b8d7 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:34.175207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62ea79-29a3-474b-b0db-65644e1b224b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:34.176397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62ea79-29a3-474b-b0db-65644e1b224b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.176687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62ea79-29a3-474b-b0db-65644e1b224b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.176967 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62ea79-29a3-474b-b0db-65644e1b224b None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.177814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62ea79-29a3-474b-b0db-65644e1b224b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:34.178124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62ea79-29a3-474b-b0db-65644e1b224b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:34.179545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62ea79-29a3-474b-b0db-65644e1b224b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:34.200369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 366/728] 10.222.0.44 () {56 vars in 1133 bytes} [Mon May 9 04:13:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:34.209617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62ea79-29a3-474b-b0db-65644e1b224b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:34.321579 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 363/729] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 04:13:34.328184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62ea79-29a3-474b-b0db-65644e1b224b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:34.356208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2e435-304b-44a0-a3eb-cc55e443ec92 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:34.370697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b27706-7db5-4def-8091-13c0499cf11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:34.371514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b27706-7db5-4def-8091-13c0499cf11d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.371796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b27706-7db5-4def-8091-13c0499cf11d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.372095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b27706-7db5-4def-8091-13c0499cf11d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.395717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b27706-7db5-4def-8091-13c0499cf11d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:34.397197 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b27706-7db5-4def-8091-13c0499cf11d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-62b27706-7db5-4def-8091-13c0499cf11d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:34.432021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:34.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:34.435117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:34.435117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:34.435117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:34.435117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:34.435117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 367/730] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:34.441860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b27706-7db5-4def-8091-13c0499cf11d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:34.523665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2e435-304b-44a0-a3eb-cc55e443ec92 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:34.560779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120512c-5697-4678-8a0e-ba39b5cc699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:34.562630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120512c-5697-4678-8a0e-ba39b5cc699f None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.562630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120512c-5697-4678-8a0e-ba39b5cc699f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.562857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120512c-5697-4678-8a0e-ba39b5cc699f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.565911 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b27706-7db5-4def-8091-13c0499cf11d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:34.574994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120512c-5697-4678-8a0e-ba39b5cc699f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-6120512c-5697-4678-8a0e-ba39b5cc699f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:34.584489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:34.585512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:34.587055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:34.587055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:34.587055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:34.587055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:34.587055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:34.587055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:34.587055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:34.587055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:34.587055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:34.587055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:34.587055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:34.587055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:34.587055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 364/731] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:13:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:34.587760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd3b92-0771-4e91-a91d-7562c02d47b2 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.587760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd3b92-0771-4e91-a91d-7562c02d47b2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.588114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd3b92-0771-4e91-a91d-7562c02d47b2 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.594082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 365/732] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:34.594082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd3b92-0771-4e91-a91d-7562c02d47b2 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:34.614634 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2b20c-a050-4c85-a19b-40b9c18a4929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:34.616037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2b20c-a050-4c85-a19b-40b9c18a4929 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.616682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2b20c-a050-4c85-a19b-40b9c18a4929 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.617307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2b20c-a050-4c85-a19b-40b9c18a4929 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.618302 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2b20c-a050-4c85-a19b-40b9c18a4929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:34.618833 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2b20c-a050-4c85-a19b-40b9c18a4929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:34.620457 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2b20c-a050-4c85-a19b-40b9c18a4929 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:34.633266 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 368/733] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:34.635898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef46638-308f-45c0-b021-b6fca755b806 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.636602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef46638-308f-45c0-b021-b6fca755b806 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.637598 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef46638-308f-45c0-b021-b6fca755b806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.656797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-cef46638-308f-45c0-b021-b6fca755b806 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:34.658142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-cef46638-308f-45c0-b021-b6fca755b806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:34.699530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd3b92-0771-4e91-a91d-7562c02d47b2 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:34.726533 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 369/734] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:34.727524 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef46638-308f-45c0-b021-b6fca755b806 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:34.753194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836378c-5c51-49c7-a0e3-2aa1449db6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:34.753913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836378c-5c51-49c7-a0e3-2aa1449db6c5 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.754202 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836378c-5c51-49c7-a0e3-2aa1449db6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.754509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836378c-5c51-49c7-a0e3-2aa1449db6c5 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.755271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836378c-5c51-49c7-a0e3-2aa1449db6c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:34.755580 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836378c-5c51-49c7-a0e3-2aa1449db6c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:34.757002 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836378c-5c51-49c7-a0e3-2aa1449db6c5 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:34.767021 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 366/735] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:34.770140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a728f-a39b-4660-9aff-3619b1d33070 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.770423 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a728f-a39b-4660-9aff-3619b1d33070 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.770697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a728f-a39b-4660-9aff-3619b1d33070 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.772401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 367/736] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:34.773565 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f282-f919-494f-ad21-794c56ab81e7 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.773911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f282-f919-494f-ad21-794c56ab81e7 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.774135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64f282-f919-494f-ad21-794c56ab81e7 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.775600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 368/737] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:34.776425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64f282-f919-494f-ad21-794c56ab81e7 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:34.831597 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef46638-308f-45c0-b021-b6fca755b806 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:34.862891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed668f5-0094-4a80-8251-f0528aadb36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:34.863670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed668f5-0094-4a80-8251-f0528aadb36e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.863990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed668f5-0094-4a80-8251-f0528aadb36e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.864248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed668f5-0094-4a80-8251-f0528aadb36e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.870455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed668f5-0094-4a80-8251-f0528aadb36e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:34.871978 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed668f5-0094-4a80-8251-f0528aadb36e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:34.874555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64f282-f919-494f-ad21-794c56ab81e7 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-3ed668f5-0094-4a80-8251-f0528aadb36e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:34.885335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:34.886271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:34.887180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:34.887180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:34.887180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:34.887180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:34.887180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:34.887180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:34.887180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:34.887180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:34.887180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 370/738] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:34.888774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed668f5-0094-4a80-8251-f0528aadb36e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:34.911725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a83b9-9e98-4d4f-b208-1f7f922a07ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:34.912548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a83b9-9e98-4d4f-b208-1f7f922a07ac None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.912830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a83b9-9e98-4d4f-b208-1f7f922a07ac None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.913104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a83b9-9e98-4d4f-b208-1f7f922a07ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.923164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a83b9-9e98-4d4f-b208-1f7f922a07ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-a40a83b9-9e98-4d4f-b208-1f7f922a07ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:34.930721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:34.931623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:34.932697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:34.932697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:34.932697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:34.932697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:34.932697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:34.932697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:34.932697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:34.932697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:34.932697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:34.932697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:34.932697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:34.932697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:34.932697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 369/739] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:34.933401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a6431-86d9-4eb3-a9bd-a9b7cea12602 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:34.933401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a6431-86d9-4eb3-a9bd-a9b7cea12602 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:34.933721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a6431-86d9-4eb3-a9bd-a9b7cea12602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:34.949706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-899a6431-86d9-4eb3-a9bd-a9b7cea12602 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:34.951174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-899a6431-86d9-4eb3-a9bd-a9b7cea12602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:34.986766 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed668f5-0094-4a80-8251-f0528aadb36e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:35.012712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 370/740] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:35.014272 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4ea6d-846b-49e2-92cb-13f46ce38afc None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:35.014557 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4ea6d-846b-49e2-92cb-13f46ce38afc None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:35.014889 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4ea6d-846b-49e2-92cb-13f46ce38afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:35.017631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064409a-7504-4af3-8eb7-fe365ed72518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:35.018412 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064409a-7504-4af3-8eb7-fe365ed72518 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:35.018692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064409a-7504-4af3-8eb7-fe365ed72518 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:35.018967 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064409a-7504-4af3-8eb7-fe365ed72518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:35.029889 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064409a-7504-4af3-8eb7-fe365ed72518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:35.031254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064409a-7504-4af3-8eb7-fe365ed72518 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:35.031637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-84d4ea6d-846b-49e2-92cb-13f46ce38afc None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:35.032832 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-84d4ea6d-846b-49e2-92cb-13f46ce38afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-d064409a-7504-4af3-8eb7-fe365ed72518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:35.049216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:35.050139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:35.051037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:35.051037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:35.051037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:35.051037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:35.052186 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 371/741] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:35.052863 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064409a-7504-4af3-8eb7-fe365ed72518 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:35.097841 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 371/742] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 04:13:35.098667 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4ea6d-846b-49e2-92cb-13f46ce38afc None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:35.148206 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064409a-7504-4af3-8eb7-fe365ed72518 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:35.179158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77ae62-c610-438b-aa80-9a6ff928064b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:35.179816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77ae62-c610-438b-aa80-9a6ff928064b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:35.180124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77ae62-c610-438b-aa80-9a6ff928064b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:35.180399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77ae62-c610-438b-aa80-9a6ff928064b None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:35.181178 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77ae62-c610-438b-aa80-9a6ff928064b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:35.181485 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77ae62-c610-438b-aa80-9a6ff928064b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:35.182892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77ae62-c610-438b-aa80-9a6ff928064b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:35.190818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4ea6d-846b-49e2-92cb-13f46ce38afc None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:35.194392 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 372/743] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:35.194968 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77ae62-c610-438b-aa80-9a6ff928064b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:35.224472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6168eff-0521-45c0-a6e0-db4dfa8fabd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:35.225244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6168eff-0521-45c0-a6e0-db4dfa8fabd4 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:35.225526 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6168eff-0521-45c0-a6e0-db4dfa8fabd4 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:35.225798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6168eff-0521-45c0-a6e0-db4dfa8fabd4 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:35.226564 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6168eff-0521-45c0-a6e0-db4dfa8fabd4 None admin] RBAC: Adding query filter params (name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:35.226864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6168eff-0521-45c0-a6e0-db4dfa8fabd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:35.228234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6168eff-0521-45c0-a6e0-db4dfa8fabd4 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:35.235967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 372/744] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:35.236723 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6168eff-0521-45c0-a6e0-db4dfa8fabd4 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:35.288379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77ae62-c610-438b-aa80-9a6ff928064b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:35.319623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea46db2-4579-403d-874a-525617c713dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:35.320303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea46db2-4579-403d-874a-525617c713dd None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:35.320585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea46db2-4579-403d-874a-525617c713dd None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:35.320858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea46db2-4579-403d-874a-525617c713dd None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:35.321565 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea46db2-4579-403d-874a-525617c713dd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:35.322908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea46db2-4579-403d-874a-525617c713dd None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:35.329571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6168eff-0521-45c0-a6e0-db4dfa8fabd4 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:35.337764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 373/745] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:13:35 2022] POST /identity/v3/services => generated 251 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:13:35.338797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea46db2-4579-403d-874a-525617c713dd None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:35.370008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b768f47-841a-4006-9fea-0804478b1b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:35.370698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b768f47-841a-4006-9fea-0804478b1b7a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:35.370976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b768f47-841a-4006-9fea-0804478b1b7a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:35.371256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b768f47-841a-4006-9fea-0804478b1b7a None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:35.372025 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b768f47-841a-4006-9fea-0804478b1b7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:35.372335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b768f47-841a-4006-9fea-0804478b1b7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:35.374247 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b768f47-841a-4006-9fea-0804478b1b7a None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:35.386898 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 373/746] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:35.387566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b768f47-841a-4006-9fea-0804478b1b7a None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:35.449652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea46db2-4579-403d-874a-525617c713dd None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:35.485091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e5093-0fa5-4d4e-aabc-19ebf1686964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:35.485763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e5093-0fa5-4d4e-aabc-19ebf1686964 None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:35.486049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e5093-0fa5-4d4e-aabc-19ebf1686964 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:35.486381 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e5093-0fa5-4d4e-aabc-19ebf1686964 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:35.487109 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e5093-0fa5-4d4e-aabc-19ebf1686964 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:35.488548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e5093-0fa5-4d4e-aabc-19ebf1686964 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:35.492532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b768f47-841a-4006-9fea-0804478b1b7a None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:35.515923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 374/747] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:13:35 2022] POST /identity/v3/services => generated 240 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:13:35.515923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e5093-0fa5-4d4e-aabc-19ebf1686964 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:35.562211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841a88c-9ce8-44b4-8c0e-3e4983add2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:35.563058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841a88c-9ce8-44b4-8c0e-3e4983add2f4 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:35.563334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841a88c-9ce8-44b4-8c0e-3e4983add2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:35.563605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841a88c-9ce8-44b4-8c0e-3e4983add2f4 None admin] PATH_INFO: `/v3/projects/3322b00723974bd99e837d447bac22d1/users/faf6107735424ff7901614953e7df403/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:35.565752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841a88c-9ce8-44b4-8c0e-3e4983add2f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3322b00723974bd99e837d447bac22d1, user_id=faf6107735424ff7901614953e7df403, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:35.567233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841a88c-9ce8-44b4-8c0e-3e4983add2f4 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:35.583518 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 374/748] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:13:35 2022] PUT /identity/v3/projects/3322b00723974bd99e837d447bac22d1/users/faf6107735424ff7901614953e7df403/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:13:35.584168 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841a88c-9ce8-44b4-8c0e-3e4983add2f4 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:35.620309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e5093-0fa5-4d4e-aabc-19ebf1686964 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:35.658610 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3fbff1-d87b-45c1-8084-3552aceedd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:35.659454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fbff1-d87b-45c1-8084-3552aceedd9e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:35.659732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fbff1-d87b-45c1-8084-3552aceedd9e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:35.660047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fbff1-d87b-45c1-8084-3552aceedd9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:35.673136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3fbff1-d87b-45c1-8084-3552aceedd9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:35.674628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3fbff1-d87b-45c1-8084-3552aceedd9e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-0f3fbff1-d87b-45c1-8084-3552aceedd9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:35.695337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:35.696545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:35.697715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:35.697715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:35.697715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:35.697715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:35.697962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841a88c-9ce8-44b4-8c0e-3e4983add2f4 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:35.698042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 375/749] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:35.709011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3fbff1-d87b-45c1-8084-3552aceedd9e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:35.765206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcef946-536c-483a-9c9c-37d4c2173c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:35.766173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcef946-536c-483a-9c9c-37d4c2173c78 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:35.766560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcef946-536c-483a-9c9c-37d4c2173c78 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:35.766928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcef946-536c-483a-9c9c-37d4c2173c78 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:35.776385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcef946-536c-483a-9c9c-37d4c2173c78 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:35.778005 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcef946-536c-483a-9c9c-37d4c2173c78 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-1dcef946-536c-483a-9c9c-37d4c2173c78 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:35.796593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:35.797613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:35.799040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:35.799040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:35.799040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:35.799040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:35.799040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:35.799040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:35.799040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:35.799040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:35.799040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 04:13:35.799040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:35.799040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 375/750] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:13:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:35.807764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcef946-536c-483a-9c9c-37d4c2173c78 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:35.824807 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3fbff1-d87b-45c1-8084-3552aceedd9e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:35.862358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab27eaa-8ff2-42c5-93c6-b31236c2c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:35.863038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab27eaa-8ff2-42c5-93c6-b31236c2c53c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:35.863320 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab27eaa-8ff2-42c5-93c6-b31236c2c53c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:35.863593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab27eaa-8ff2-42c5-93c6-b31236c2c53c None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:35.864357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab27eaa-8ff2-42c5-93c6-b31236c2c53c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:35.864661 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab27eaa-8ff2-42c5-93c6-b31236c2c53c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:35.866019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab27eaa-8ff2-42c5-93c6-b31236c2c53c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:35.878833 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 376/751] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:35.885852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab27eaa-8ff2-42c5-93c6-b31236c2c53c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:35.958608 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcef946-536c-483a-9c9c-37d4c2173c78 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:35.990193 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab27eaa-8ff2-42c5-93c6-b31236c2c53c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:36.012627 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157c36e-2d04-4997-8322-99f969f8bc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:36.014763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157c36e-2d04-4997-8322-99f969f8bc0f None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:36.015046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157c36e-2d04-4997-8322-99f969f8bc0f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:36.015317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157c36e-2d04-4997-8322-99f969f8bc0f None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:36.016256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157c36e-2d04-4997-8322-99f969f8bc0f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:36.016567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157c36e-2d04-4997-8322-99f969f8bc0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:36.018055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157c36e-2d04-4997-8322-99f969f8bc0f None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:36.026659 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f36b9-17fb-427d-83a1-2e3314d76a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:36.028182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f36b9-17fb-427d-83a1-2e3314d76a23 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:36.029207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f36b9-17fb-427d-83a1-2e3314d76a23 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:36.029686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f36b9-17fb-427d-83a1-2e3314d76a23 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:36.037616 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f36b9-17fb-427d-83a1-2e3314d76a23 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:36.039371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f36b9-17fb-427d-83a1-2e3314d76a23 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:36.041126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 376/752] 10.222.0.44 () {56 vars in 1135 bytes} [Mon May 9 04:13:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:36.046811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157c36e-2d04-4997-8322-99f969f8bc0f None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-694f36b9-17fb-427d-83a1-2e3314d76a23 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:36.056620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:36.057621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:36.058573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:36.058573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:36.058573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:36.058573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:36.058573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:36.058573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:36.058573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:36.058573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:36.058573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 04:13:36.058573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:36.065395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 377/753] 10.222.0.44 () {56 vars in 1117 bytes} [Mon May 9 04:13:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:36.066897 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f36b9-17fb-427d-83a1-2e3314d76a23 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:36.150162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157c36e-2d04-4997-8322-99f969f8bc0f None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:36.201127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f36b9-17fb-427d-83a1-2e3314d76a23 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:36.217116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afb4c8-1e32-453d-b6f8-9a5514d0de03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:36.217852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afb4c8-1e32-453d-b6f8-9a5514d0de03 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:36.218374 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afb4c8-1e32-453d-b6f8-9a5514d0de03 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:36.218374 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afb4c8-1e32-453d-b6f8-9a5514d0de03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:36.238770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973c4e6-9460-4f5f-8b50-256e915aabda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:36.239792 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973c4e6-9460-4f5f-8b50-256e915aabda None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:36.240412 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973c4e6-9460-4f5f-8b50-256e915aabda None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:36.240887 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973c4e6-9460-4f5f-8b50-256e915aabda None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:36.241924 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973c4e6-9460-4f5f-8b50-256e915aabda None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:36.242429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973c4e6-9460-4f5f-8b50-256e915aabda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:36.244087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973c4e6-9460-4f5f-8b50-256e915aabda None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:36.266219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 378/754] 10.222.0.44 () {56 vars in 1149 bytes} [Mon May 9 04:13:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:36.267820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973c4e6-9460-4f5f-8b50-256e915aabda None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:36.319127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afb4c8-1e32-453d-b6f8-9a5514d0de03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:36.321135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afb4c8-1e32-453d-b6f8-9a5514d0de03 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:36.425293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973c4e6-9460-4f5f-8b50-256e915aabda None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:36.489515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469d65e-8ada-420c-a068-5aaad0b1554e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:36.490523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469d65e-8ada-420c-a068-5aaad0b1554e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:36.491737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469d65e-8ada-420c-a068-5aaad0b1554e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:36.492477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469d65e-8ada-420c-a068-5aaad0b1554e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:36.513581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 377/755] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 04:13:36.521634 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afb4c8-1e32-453d-b6f8-9a5514d0de03 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:36.583522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469d65e-8ada-420c-a068-5aaad0b1554e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:36.585013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469d65e-8ada-420c-a068-5aaad0b1554e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:36.696222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 379/756] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 04:13:36.701954 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469d65e-8ada-420c-a068-5aaad0b1554e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:36.730207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afb4c8-1e32-453d-b6f8-9a5514d0de03 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:36.813149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb62f10-5844-4f8b-9aba-5553ba768dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:36.813954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb62f10-5844-4f8b-9aba-5553ba768dc6 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:36.814233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb62f10-5844-4f8b-9aba-5553ba768dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:36.814501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb62f10-5844-4f8b-9aba-5553ba768dc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:36.837769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb62f10-5844-4f8b-9aba-5553ba768dc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:36.839213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb62f10-5844-4f8b-9aba-5553ba768dc6 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:36.847159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469d65e-8ada-420c-a068-5aaad0b1554e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-3cb62f10-5844-4f8b-9aba-5553ba768dc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:36.878945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:36.879919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:36.880821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:36.880821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:36.880821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:36.880821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:36.880821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 378/757] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:36.883847 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6243d30-40c1-4f69-9239-069b3659582a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:36.884686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6243d30-40c1-4f69-9239-069b3659582a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:36.884969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6243d30-40c1-4f69-9239-069b3659582a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:36.885246 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6243d30-40c1-4f69-9239-069b3659582a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:36.892271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb62f10-5844-4f8b-9aba-5553ba768dc6 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:36.915531 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6243d30-40c1-4f69-9239-069b3659582a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:36.917011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6243d30-40c1-4f69-9239-069b3659582a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-a6243d30-40c1-4f69-9239-069b3659582a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:36.945090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:36.947060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:36.948207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:36.948207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:36.948207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:36.948207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:36.948207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 380/758] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:36.956874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6243d30-40c1-4f69-9239-069b3659582a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:37.016810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb62f10-5844-4f8b-9aba-5553ba768dc6 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:37.057191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e0527-0431-44cb-adb7-d9b14d314323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:37.057869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e0527-0431-44cb-adb7-d9b14d314323 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:37.058150 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e0527-0431-44cb-adb7-d9b14d314323 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:37.058424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e0527-0431-44cb-adb7-d9b14d314323 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:37.059166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e0527-0431-44cb-adb7-d9b14d314323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:37.059472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e0527-0431-44cb-adb7-d9b14d314323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:37.060923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e0527-0431-44cb-adb7-d9b14d314323 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:37.085255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 379/759] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:37.093390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e0527-0431-44cb-adb7-d9b14d314323 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:37.127680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6243d30-40c1-4f69-9239-069b3659582a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:37.161983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bd15a-fc1b-4072-ade9-b8eb760552dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:37.162659 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bd15a-fc1b-4072-ade9-b8eb760552dc None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:37.162942 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bd15a-fc1b-4072-ade9-b8eb760552dc None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:37.163223 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bd15a-fc1b-4072-ade9-b8eb760552dc None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:37.163968 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bd15a-fc1b-4072-ade9-b8eb760552dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:37.164279 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bd15a-fc1b-4072-ade9-b8eb760552dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:37.165743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bd15a-fc1b-4072-ade9-b8eb760552dc None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:37.177931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 381/760] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:37.184139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bd15a-fc1b-4072-ade9-b8eb760552dc None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:37.249294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e0527-0431-44cb-adb7-d9b14d314323 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:37.284101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad55a3-2f39-41f8-b869-27a250d18513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:37.284906 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad55a3-2f39-41f8-b869-27a250d18513 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:37.285189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad55a3-2f39-41f8-b869-27a250d18513 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:37.285468 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad55a3-2f39-41f8-b869-27a250d18513 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:37.292385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad55a3-2f39-41f8-b869-27a250d18513 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:37.293836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad55a3-2f39-41f8-b869-27a250d18513 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:37.294020 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bd15a-fc1b-4072-ade9-b8eb760552dc None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-c8ad55a3-2f39-41f8-b869-27a250d18513 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:37.309397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:37.312048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:37.314192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:37.314192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:37.314192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:37.314192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:37.314192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:37.314192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:37.314192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:37.314192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:37.316387 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 380/761] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:37.319916 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad55a3-2f39-41f8-b869-27a250d18513 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:37.329451 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d727d-97ec-4194-ba0d-fe7a0caa5494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:37.330254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d727d-97ec-4194-ba0d-fe7a0caa5494 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:37.330540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d727d-97ec-4194-ba0d-fe7a0caa5494 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:37.330820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d727d-97ec-4194-ba0d-fe7a0caa5494 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:37.337641 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d727d-97ec-4194-ba0d-fe7a0caa5494 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:37.339079 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d727d-97ec-4194-ba0d-fe7a0caa5494 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-a32d727d-97ec-4194-ba0d-fe7a0caa5494 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:37.353811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:37.355707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:37.357049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:37.357049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:37.357049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:37.357049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:37.357049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:37.357049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:37.357049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:37.357049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:37.357049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 382/762] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:37.368959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d727d-97ec-4194-ba0d-fe7a0caa5494 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:37.421532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad55a3-2f39-41f8-b869-27a250d18513 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:37.456502 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e786d0-9bb8-4983-9c6d-43aa012ad2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:37.457168 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e786d0-9bb8-4983-9c6d-43aa012ad2d3 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:37.457452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e786d0-9bb8-4983-9c6d-43aa012ad2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:37.457728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e786d0-9bb8-4983-9c6d-43aa012ad2d3 None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:37.458517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e786d0-9bb8-4983-9c6d-43aa012ad2d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:37.458826 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e786d0-9bb8-4983-9c6d-43aa012ad2d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:37.460284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e786d0-9bb8-4983-9c6d-43aa012ad2d3 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:37.472520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 381/763] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:37.478210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e786d0-9bb8-4983-9c6d-43aa012ad2d3 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:37.507313 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d727d-97ec-4194-ba0d-fe7a0caa5494 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:37.565744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f17ba-4c20-425b-b1b0-ab46c9efea9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:37.566350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f17ba-4c20-425b-b1b0-ab46c9efea9e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:37.566683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f17ba-4c20-425b-b1b0-ab46c9efea9e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:37.566963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f17ba-4c20-425b-b1b0-ab46c9efea9e None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:37.567754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f17ba-4c20-425b-b1b0-ab46c9efea9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:37.568086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f17ba-4c20-425b-b1b0-ab46c9efea9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:37.569509 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f17ba-4c20-425b-b1b0-ab46c9efea9e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:37.586076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e786d0-9bb8-4983-9c6d-43aa012ad2d3 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:37.590200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 383/764] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:37.596505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f17ba-4c20-425b-b1b0-ab46c9efea9e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:37.629779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69321b7-4230-4142-8b23-0c46885156c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:37.630446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69321b7-4230-4142-8b23-0c46885156c0 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:37.630730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69321b7-4230-4142-8b23-0c46885156c0 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:37.631010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69321b7-4230-4142-8b23-0c46885156c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:37.631849 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69321b7-4230-4142-8b23-0c46885156c0 None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=c46e7cc6407f44f08a659288e3ad0767) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:37.633525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69321b7-4230-4142-8b23-0c46885156c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:37.634975 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69321b7-4230-4142-8b23-0c46885156c0 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:37.646421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 382/765] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:37 2022] GET /identity/v3/role_assignments?user.id=c46e7cc6407f44f08a659288e3ad0767&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=e59113c0366049e889df3f55eed90c72 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:37.727872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f17ba-4c20-425b-b1b0-ab46c9efea9e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:37.787116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2940304-c0f3-4e45-8637-f09fad7a0d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:37.788190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2940304-c0f3-4e45-8637-f09fad7a0d77 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:37.788681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2940304-c0f3-4e45-8637-f09fad7a0d77 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:37.789177 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2940304-c0f3-4e45-8637-f09fad7a0d77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:37.790205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2940304-c0f3-4e45-8637-f09fad7a0d77 None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=ee0055c75b91471a99d9bb18248839ea) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:37.790736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2940304-c0f3-4e45-8637-f09fad7a0d77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:37.796639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2940304-c0f3-4e45-8637-f09fad7a0d77 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:37.803188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 384/766] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:37 2022] GET /identity/v3/role_assignments?user.id=ee0055c75b91471a99d9bb18248839ea&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:38.141015 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0382d-f45e-40a6-9bef-f9c35ef4295f None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:38.141515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0382d-f45e-40a6-9bef-f9c35ef4295f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:38.141515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0382d-f45e-40a6-9bef-f9c35ef4295f None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:38.143180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 383/767] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:38.149326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6dc28-4158-4c82-89f8-f4d8488f4ff2 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:38.149834 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6dc28-4158-4c82-89f8-f4d8488f4ff2 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:38.149910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6dc28-4158-4c82-89f8-f4d8488f4ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:38.167576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-edf6dc28-4158-4c82-89f8-f4d8488f4ff2 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:38.168849 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-edf6dc28-4158-4c82-89f8-f4d8488f4ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:38.241842 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 385/768] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:13:38.265369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0382d-f45e-40a6-9bef-f9c35ef4295f None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:38.315142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35699b-db15-43d6-a1ed-0060258bc26b None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:38.315457 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35699b-db15-43d6-a1ed-0060258bc26b None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:38.315709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35699b-db15-43d6-a1ed-0060258bc26b None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:38.317272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 386/769] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:13:38.322687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ad0c6-2f09-41e7-bdd8-af239f4c03b8 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:38.323308 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ad0c6-2f09-41e7-bdd8-af239f4c03b8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:38.323787 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ad0c6-2f09-41e7-bdd8-af239f4c03b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:38.354006 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-a14ad0c6-2f09-41e7-bdd8-af239f4c03b8 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:38.355591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a14ad0c6-2f09-41e7-bdd8-af239f4c03b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:38.460073 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 387/770] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:13:38.461387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fced38-4899-413a-be7e-c4211356a17f None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:38.461680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fced38-4899-413a-be7e-c4211356a17f None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:38.461958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fced38-4899-413a-be7e-c4211356a17f None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:38.463512 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 388/771] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:38.470506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f26be3-8bfe-4fae-8bba-f57c0d0c0c99 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:38.470797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f26be3-8bfe-4fae-8bba-f57c0d0c0c99 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:38.471080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f26be3-8bfe-4fae-8bba-f57c0d0c0c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:38.495484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-77f26be3-8bfe-4fae-8bba-f57c0d0c0c99 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:38.496686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-77f26be3-8bfe-4fae-8bba-f57c0d0c0c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:38.506656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0382d-f45e-40a6-9bef-f9c35ef4295f None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:38.547205 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b30fc2-6fe7-4777-af00-184639198de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:38.548386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b30fc2-6fe7-4777-af00-184639198de1 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:38.548675 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b30fc2-6fe7-4777-af00-184639198de1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:38.549272 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b30fc2-6fe7-4777-af00-184639198de1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:38.549949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b30fc2-6fe7-4777-af00-184639198de1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:38.551915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b30fc2-6fe7-4777-af00-184639198de1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-95b30fc2-6fe7-4777-af00-184639198de1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:38.567858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:38.570808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:38.571659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 384/772] 10.222.0.44 () {56 vars in 1129 bytes} [Mon May 9 04:13:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:38.572551 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b30fc2-6fe7-4777-af00-184639198de1 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:38.584788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 389/773] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:13:38.592173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f26be3-8bfe-4fae-8bba-f57c0d0c0c99 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:38.685843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f26be3-8bfe-4fae-8bba-f57c0d0c0c99 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:38.697693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b30fc2-6fe7-4777-af00-184639198de1 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:38.720474 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345a479-29f3-4e9c-bd3f-30a1a81278a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:38.721188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345a479-29f3-4e9c-bd3f-30a1a81278a2 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:38.721485 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345a479-29f3-4e9c-bd3f-30a1a81278a2 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:38.721793 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345a479-29f3-4e9c-bd3f-30a1a81278a2 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:38.722513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345a479-29f3-4e9c-bd3f-30a1a81278a2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:38.722821 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345a479-29f3-4e9c-bd3f-30a1a81278a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:38.724393 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345a479-29f3-4e9c-bd3f-30a1a81278a2 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:38.731989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 390/774] 10.222.0.44 () {56 vars in 1124 bytes} [Mon May 9 04:13:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:38.733340 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345a479-29f3-4e9c-bd3f-30a1a81278a2 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:38.737139 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1403cd-1688-42ca-be83-be67f7fc268d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:38.738178 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1403cd-1688-42ca-be83-be67f7fc268d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:38.738698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1403cd-1688-42ca-be83-be67f7fc268d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:38.739180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1403cd-1688-42ca-be83-be67f7fc268d None admin] PATH_INFO: `/v3/services/network` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:38.740092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1403cd-1688-42ca-be83-be67f7fc268d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:38.741703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1403cd-1688-42ca-be83-be67f7fc268d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-7e1403cd-1688-42ca-be83-be67f7fc268d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:38.755451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:38.756534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 04:13:38.757424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:38.760438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 385/775] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:38 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:38.761373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1403cd-1688-42ca-be83-be67f7fc268d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:38.852303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345a479-29f3-4e9c-bd3f-30a1a81278a2 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:38.862608 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1403cd-1688-42ca-be83-be67f7fc268d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:38.887188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52f2c0-8b78-49e6-b41e-c2f388bd5eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:38.888815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52f2c0-8b78-49e6-b41e-c2f388bd5eaf None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:38.889431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52f2c0-8b78-49e6-b41e-c2f388bd5eaf None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:38.889942 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52f2c0-8b78-49e6-b41e-c2f388bd5eaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:38.897616 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2d3bd-4357-45f8-9966-ffe7770d40c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:38.898504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2d3bd-4357-45f8-9966-ffe7770d40c3 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:38.898665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2d3bd-4357-45f8-9966-ffe7770d40c3 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:38.899041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2d3bd-4357-45f8-9966-ffe7770d40c3 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:38.899823 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2d3bd-4357-45f8-9966-ffe7770d40c3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:38.900230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2d3bd-4357-45f8-9966-ffe7770d40c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:38.901676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2d3bd-4357-45f8-9966-ffe7770d40c3 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:38.902231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52f2c0-8b78-49e6-b41e-c2f388bd5eaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:38.912188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 386/776] 10.222.0.44 () {56 vars in 1124 bytes} [Mon May 9 04:13:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:38.912188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2d3bd-4357-45f8-9966-ffe7770d40c3 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-da52f2c0-8b78-49e6-b41e-c2f388bd5eaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:38.916129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:38.917134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:38.917979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:38.917979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:38.917979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:38.917979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:38.917979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:38.917979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:38.917979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:38.917979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:38.917979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:38.917979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:38.917979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:38.917979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:38.920822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 391/777] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:13:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:38.921803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52f2c0-8b78-49e6-b41e-c2f388bd5eaf None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:39.012419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2d3bd-4357-45f8-9966-ffe7770d40c3 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:39.022423 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52f2c0-8b78-49e6-b41e-c2f388bd5eaf None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:39.046710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35da37-9068-4860-b829-3efcd713895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:39.047552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35da37-9068-4860-b829-3efcd713895c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:39.047923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35da37-9068-4860-b829-3efcd713895c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:39.048341 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35da37-9068-4860-b829-3efcd713895c None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:39.049149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35da37-9068-4860-b829-3efcd713895c None admin] RBAC: Adding query filter params (name=network) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:39.049566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35da37-9068-4860-b829-3efcd713895c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:39.051064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35da37-9068-4860-b829-3efcd713895c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:39.055849 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 387/778] 10.222.0.44 () {56 vars in 1112 bytes} [Mon May 9 04:13:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:39.056848 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35da37-9068-4860-b829-3efcd713895c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:39.057350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484355-1d6b-421a-bc60-8a1c37a226de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:39.058170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484355-1d6b-421a-bc60-8a1c37a226de None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:39.058630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484355-1d6b-421a-bc60-8a1c37a226de None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:39.059050 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484355-1d6b-421a-bc60-8a1c37a226de None admin] PATH_INFO: `/v3/endpoints` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:39.060666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484355-1d6b-421a-bc60-8a1c37a226de None admin] RBAC: Adding query filter params (interface=public, service_id=6a24e898642b46f78f4be73a491555d3, region_id=RegionOne) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:39.061200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484355-1d6b-421a-bc60-8a1c37a226de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:39.062739 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484355-1d6b-421a-bc60-8a1c37a226de None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:39.076134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 392/779] 10.222.0.44 () {56 vars in 1251 bytes} [Mon May 9 04:13:38 2022] GET /identity/v3/endpoints?service_id=6a24e898642b46f78f4be73a491555d3&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:39.077223 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484355-1d6b-421a-bc60-8a1c37a226de None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:39.158651 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35da37-9068-4860-b829-3efcd713895c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:39.180164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484355-1d6b-421a-bc60-8a1c37a226de None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:39.192745 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2cbc6-1a95-4680-a485-009067e3462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:39.193475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2cbc6-1a95-4680-a485-009067e3462f None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:39.193699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2cbc6-1a95-4680-a485-009067e3462f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:39.193984 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2cbc6-1a95-4680-a485-009067e3462f None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:39.194738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2cbc6-1a95-4680-a485-009067e3462f None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:39.195047 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2cbc6-1a95-4680-a485-009067e3462f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:39.196462 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2cbc6-1a95-4680-a485-009067e3462f None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:39.206486 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 388/780] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:39.207375 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2cbc6-1a95-4680-a485-009067e3462f None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:39.220720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd322c-8855-43b3-856a-32a8fda2bb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:39.221787 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd322c-8855-43b3-856a-32a8fda2bb14 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:39.222277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd322c-8855-43b3-856a-32a8fda2bb14 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:39.222780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd322c-8855-43b3-856a-32a8fda2bb14 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:39.223773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd322c-8855-43b3-856a-32a8fda2bb14 None admin] RBAC: Adding query filter params (type=network) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:39.224358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd322c-8855-43b3-856a-32a8fda2bb14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:39.225937 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd322c-8855-43b3-856a-32a8fda2bb14 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:39.231769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 393/781] 10.222.0.44 () {56 vars in 1112 bytes} [Mon May 9 04:13:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:39.233067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd322c-8855-43b3-856a-32a8fda2bb14 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:39.313432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2cbc6-1a95-4680-a485-009067e3462f None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:39.334520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd322c-8855-43b3-856a-32a8fda2bb14 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:39.348341 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb59996-7045-4df9-8ffe-9ae5c9738406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:39.349201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb59996-7045-4df9-8ffe-9ae5c9738406 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:39.349340 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb59996-7045-4df9-8ffe-9ae5c9738406 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:39.349579 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb59996-7045-4df9-8ffe-9ae5c9738406 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:39.350319 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb59996-7045-4df9-8ffe-9ae5c9738406 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:39.351688 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb59996-7045-4df9-8ffe-9ae5c9738406 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:39.364076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 389/782] 10.222.0.44 () {56 vars in 1087 bytes} [Mon May 9 04:13:39 2022] GET /identity/v3/services => generated 772 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:39.364076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb59996-7045-4df9-8ffe-9ae5c9738406 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:39.388803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318d146-14c4-45c2-b54a-5ded11ff2736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:39.390473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318d146-14c4-45c2-b54a-5ded11ff2736 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:39.390960 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318d146-14c4-45c2-b54a-5ded11ff2736 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:39.391436 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318d146-14c4-45c2-b54a-5ded11ff2736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:39.415923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318d146-14c4-45c2-b54a-5ded11ff2736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:39.418015 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318d146-14c4-45c2-b54a-5ded11ff2736 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-2318d146-14c4-45c2-b54a-5ded11ff2736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:39.442607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:39.443564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:39.444542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:39.444542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:39.444542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:39.444542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:39.451975 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 394/783] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:39.458862 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318d146-14c4-45c2-b54a-5ded11ff2736 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:39.475924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb59996-7045-4df9-8ffe-9ae5c9738406 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:39.504587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4b4f8-265b-46df-b76c-6e05002528e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:39.505290 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4b4f8-265b-46df-b76c-6e05002528e1 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:39.505621 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4b4f8-265b-46df-b76c-6e05002528e1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:39.505817 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4b4f8-265b-46df-b76c-6e05002528e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:39.506637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4b4f8-265b-46df-b76c-6e05002528e1 None admin] RBAC: Adding query filter params (interface=public, service_id=0edfc866046a4973b6e5fcc160e5ca3f, region_id=RegionOne) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:39.507081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4b4f8-265b-46df-b76c-6e05002528e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:39.511931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4b4f8-265b-46df-b76c-6e05002528e1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:39.517163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 390/784] 10.222.0.44 () {56 vars in 1251 bytes} [Mon May 9 04:13:39 2022] GET /identity/v3/endpoints?service_id=0edfc866046a4973b6e5fcc160e5ca3f&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:39.525280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4b4f8-265b-46df-b76c-6e05002528e1 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:39.669916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318d146-14c4-45c2-b54a-5ded11ff2736 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:39.696850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4b4f8-265b-46df-b76c-6e05002528e1 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:39.713866 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b130a-30ac-457c-9d77-97e03b6a5609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:39.714353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b130a-30ac-457c-9d77-97e03b6a5609 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:39.714647 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b130a-30ac-457c-9d77-97e03b6a5609 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:39.714936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b130a-30ac-457c-9d77-97e03b6a5609 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:39.715676 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b130a-30ac-457c-9d77-97e03b6a5609 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:39.716019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b130a-30ac-457c-9d77-97e03b6a5609 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:39.717540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b130a-30ac-457c-9d77-97e03b6a5609 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:39.729967 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 395/785] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:39.736029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b130a-30ac-457c-9d77-97e03b6a5609 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:39.738985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd33a25-f4e9-49c6-9b33-52f52dafc432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:39.739938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd33a25-f4e9-49c6-9b33-52f52dafc432 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:39.746026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd33a25-f4e9-49c6-9b33-52f52dafc432 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:39.746581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd33a25-f4e9-49c6-9b33-52f52dafc432 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:39.748046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd33a25-f4e9-49c6-9b33-52f52dafc432 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:39.749755 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd33a25-f4e9-49c6-9b33-52f52dafc432 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:39.762728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 391/786] 10.222.0.44 () {56 vars in 1087 bytes} [Mon May 9 04:13:39 2022] GET /identity/v3/services => generated 772 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:39.851482 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b130a-30ac-457c-9d77-97e03b6a5609 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:39.888708 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfe20b-dbb7-4dc4-b417-9f626bdd53be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:39.889601 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfe20b-dbb7-4dc4-b417-9f626bdd53be None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:39.889885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfe20b-dbb7-4dc4-b417-9f626bdd53be None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:39.890162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfe20b-dbb7-4dc4-b417-9f626bdd53be None admin] PATH_INFO: `/v3/users/nova` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:39.897934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfe20b-dbb7-4dc4-b417-9f626bdd53be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:39.899399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfe20b-dbb7-4dc4-b417-9f626bdd53be None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-bcbfe20b-dbb7-4dc4-b417-9f626bdd53be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:39.915859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:39.916898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:39.918443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:39.918443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:39.918443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:39.918443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:39.918443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:39.918443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:39.918443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:39.918443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:39.918443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 04:13:39.918443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:39.918443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 396/787] 10.222.0.44 () {56 vars in 1093 bytes} [Mon May 9 04:13:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:39.923930 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd33a25-f4e9-49c6-9b33-52f52dafc432 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:40.064086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd33a25-f4e9-49c6-9b33-52f52dafc432 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:40.100393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6b939-373a-4fb8-84f0-0c8064c223f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:40.101072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6b939-373a-4fb8-84f0-0c8064c223f8 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:40.101360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6b939-373a-4fb8-84f0-0c8064c223f8 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:40.101635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6b939-373a-4fb8-84f0-0c8064c223f8 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:40.102485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6b939-373a-4fb8-84f0-0c8064c223f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:40.102797 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6b939-373a-4fb8-84f0-0c8064c223f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:40.104269 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6b939-373a-4fb8-84f0-0c8064c223f8 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:40.124632 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 392/788] 10.222.0.44 () {56 vars in 1133 bytes} [Mon May 9 04:13:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:40.130274 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfe20b-dbb7-4dc4-b417-9f626bdd53be None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:40.274244 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfe20b-dbb7-4dc4-b417-9f626bdd53be None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:40.323348 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ab654-ffda-4dfd-a472-58097732e68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:40.324070 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ab654-ffda-4dfd-a472-58097732e68b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:40.324352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ab654-ffda-4dfd-a472-58097732e68b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:40.324623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ab654-ffda-4dfd-a472-58097732e68b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:40.422034 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ab654-ffda-4dfd-a472-58097732e68b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:40.423638 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ab654-ffda-4dfd-a472-58097732e68b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:40.539239 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f074f-38a1-4c76-be93-7db919b99000 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:40.540200 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f074f-38a1-4c76-be93-7db919b99000 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:40.540674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f074f-38a1-4c76-be93-7db919b99000 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:40.542273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 393/789] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:13:40.550850 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 397/790] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:13:40.552503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7f5f8-8af9-4852-a5d2-a59b6f6a196b None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:40.555499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7f5f8-8af9-4852-a5d2-a59b6f6a196b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:40.555499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7f5f8-8af9-4852-a5d2-a59b6f6a196b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:40.556638 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ab654-ffda-4dfd-a472-58097732e68b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:40.579687 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-3cd7f5f8-8af9-4852-a5d2-a59b6f6a196b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:40.581284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3cd7f5f8-8af9-4852-a5d2-a59b6f6a196b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:40.683370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 394/791] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:40.688001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874a966-bb3b-4e94-8354-ff1b95d3c361 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:40.688001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874a966-bb3b-4e94-8354-ff1b95d3c361 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:40.688001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874a966-bb3b-4e94-8354-ff1b95d3c361 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:40.691868 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 395/792] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:40.698697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738a562-a214-4567-8e82-7ea4a4944146 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:40.698697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738a562-a214-4567-8e82-7ea4a4944146 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:40.698697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738a562-a214-4567-8e82-7ea4a4944146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:40.713103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c738a562-a214-4567-8e82-7ea4a4944146 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:40.714824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c738a562-a214-4567-8e82-7ea4a4944146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:40.730986 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ab654-ffda-4dfd-a472-58097732e68b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:40.781898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef258d8-82c9-4dcd-9036-8adb77b1bc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:40.783129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef258d8-82c9-4dcd-9036-8adb77b1bc03 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:40.783870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef258d8-82c9-4dcd-9036-8adb77b1bc03 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:40.784359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef258d8-82c9-4dcd-9036-8adb77b1bc03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:40.798197 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef258d8-82c9-4dcd-9036-8adb77b1bc03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:40.799844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef258d8-82c9-4dcd-9036-8adb77b1bc03 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:40.804033 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 396/793] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:13:40.805105 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738a562-a214-4567-8e82-7ea4a4944146 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-bef258d8-82c9-4dcd-9036-8adb77b1bc03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:40.835479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:40.836631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:40.837688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:40.837688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:40.837688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:40.837688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:40.839463 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 398/794] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:40.844513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef258d8-82c9-4dcd-9036-8adb77b1bc03 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:40.939003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738a562-a214-4567-8e82-7ea4a4944146 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:40.945259 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef258d8-82c9-4dcd-9036-8adb77b1bc03 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:40.979868 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8adf4-2748-4e33-b5ca-1c0ca25a037c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:40.980965 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8adf4-2748-4e33-b5ca-1c0ca25a037c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:40.981415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8adf4-2748-4e33-b5ca-1c0ca25a037c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:40.982546 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8adf4-2748-4e33-b5ca-1c0ca25a037c None admin] PATH_INFO: `/v3/roles/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:40.984092 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394904a9-bd42-428e-aa72-2029be37ff0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:40.985116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394904a9-bd42-428e-aa72-2029be37ff0a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:40.985466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394904a9-bd42-428e-aa72-2029be37ff0a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:40.986091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394904a9-bd42-428e-aa72-2029be37ff0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:40.993855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8adf4-2748-4e33-b5ca-1c0ca25a037c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:40.997276 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394904a9-bd42-428e-aa72-2029be37ff0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-e6e8adf4-2748-4e33-b5ca-1c0ca25a037c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:41.002724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:41.003854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:41.004940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:41.004940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:41.004940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:41.004940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:41.004940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:41.004940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:41.004940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:41.004940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:41.004940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:41.004940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:41.004940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:41.004940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:41.007277 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 397/795] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:41.008422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8adf4-2748-4e33-b5ca-1c0ca25a037c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-394904a9-bd42-428e-aa72-2029be37ff0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:41.012480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:41.014081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:41.015136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:41.015136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:41.015136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:41.015136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:41.015136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:41.015136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:41.015136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:41.015136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:41.015136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:41.015136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:41.015136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:41.015136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:41.017181 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 399/796] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:13:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:41.018043 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394904a9-bd42-428e-aa72-2029be37ff0a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:41.125198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394904a9-bd42-428e-aa72-2029be37ff0a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:41.139053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8adf4-2748-4e33-b5ca-1c0ca25a037c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:41.158304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440d979-5696-48f7-9cf2-dab27f21ac44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:41.159221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440d979-5696-48f7-9cf2-dab27f21ac44 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:41.159656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440d979-5696-48f7-9cf2-dab27f21ac44 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:41.160126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440d979-5696-48f7-9cf2-dab27f21ac44 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:41.161019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440d979-5696-48f7-9cf2-dab27f21ac44 None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:41.161490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440d979-5696-48f7-9cf2-dab27f21ac44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:41.162990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440d979-5696-48f7-9cf2-dab27f21ac44 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:41.172106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 400/797] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:41.173221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440d979-5696-48f7-9cf2-dab27f21ac44 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:41.182473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b296443-97fe-480c-bbbf-ce41c9b7cebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:41.183539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b296443-97fe-480c-bbbf-ce41c9b7cebf None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:41.183820 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b296443-97fe-480c-bbbf-ce41c9b7cebf None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:41.184123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b296443-97fe-480c-bbbf-ce41c9b7cebf None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:41.185467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b296443-97fe-480c-bbbf-ce41c9b7cebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:41.185887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b296443-97fe-480c-bbbf-ce41c9b7cebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:41.187613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b296443-97fe-480c-bbbf-ce41c9b7cebf None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:41.208402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 398/798] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:41.208402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b296443-97fe-480c-bbbf-ce41c9b7cebf None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:41.303039 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440d979-5696-48f7-9cf2-dab27f21ac44 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:41.316956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b296443-97fe-480c-bbbf-ce41c9b7cebf None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:41.344602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bca4d5-784c-4b02-bc54-c878d8e23f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:41.345209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bca4d5-784c-4b02-bc54-c878d8e23f8d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:41.345484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bca4d5-784c-4b02-bc54-c878d8e23f8d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:41.345760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bca4d5-784c-4b02-bc54-c878d8e23f8d None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:41.346508 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bca4d5-784c-4b02-bc54-c878d8e23f8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:41.346821 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bca4d5-784c-4b02-bc54-c878d8e23f8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:41.348217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bca4d5-784c-4b02-bc54-c878d8e23f8d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:41.353826 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23bdca-dd58-49c5-876a-2b528aa49bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:41.355166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23bdca-dd58-49c5-876a-2b528aa49bb8 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:41.355660 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23bdca-dd58-49c5-876a-2b528aa49bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:41.356177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23bdca-dd58-49c5-876a-2b528aa49bb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:41.361724 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 401/799] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:41.362603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bca4d5-784c-4b02-bc54-c878d8e23f8d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:41.369541 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23bdca-dd58-49c5-876a-2b528aa49bb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:41.371729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23bdca-dd58-49c5-876a-2b528aa49bb8 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-8b23bdca-dd58-49c5-876a-2b528aa49bb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:41.397818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:41.398787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:41.399706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:41.399706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:41.399706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:41.399706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:41.402456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 399/800] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:41.404385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23bdca-dd58-49c5-876a-2b528aa49bb8 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:41.488310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bca4d5-784c-4b02-bc54-c878d8e23f8d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:41.518945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23bdca-dd58-49c5-876a-2b528aa49bb8 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:41.525840 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee0d1d-a5fa-4be9-94c9-c91fd9844457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:41.527295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee0d1d-a5fa-4be9-94c9-c91fd9844457 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:41.527784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee0d1d-a5fa-4be9-94c9-c91fd9844457 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:41.528282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee0d1d-a5fa-4be9-94c9-c91fd9844457 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:41.537487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee0d1d-a5fa-4be9-94c9-c91fd9844457 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:41.539409 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee0d1d-a5fa-4be9-94c9-c91fd9844457 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:41.554799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c30030-70c8-4277-93c5-db6e7d8f3a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:41.555741 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c30030-70c8-4277-93c5-db6e7d8f3a21 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:41.556116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c30030-70c8-4277-93c5-db6e7d8f3a21 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:41.556493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c30030-70c8-4277-93c5-db6e7d8f3a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-44ee0d1d-a5fa-4be9-94c9-c91fd9844457 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:41.558815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:41.559683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:41.561518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:41.561518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:41.561518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:41.561518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:41.561518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:41.561518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:41.561518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:41.561518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:41.561977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 402/801] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:41.567504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee0d1d-a5fa-4be9-94c9-c91fd9844457 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:41.576444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c30030-70c8-4277-93c5-db6e7d8f3a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:41.578941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c30030-70c8-4277-93c5-db6e7d8f3a21 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-08c30030-70c8-4277-93c5-db6e7d8f3a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:41.601711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:41.602671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:41.603562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:41.603562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:41.603562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:41.603562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:41.608211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 400/802] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:41.609099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c30030-70c8-4277-93c5-db6e7d8f3a21 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:41.681463 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee0d1d-a5fa-4be9-94c9-c91fd9844457 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:41.717485 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98aca2-be4c-488f-a6f6-1680b172b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:41.718155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98aca2-be4c-488f-a6f6-1680b172b6cc None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:41.718434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98aca2-be4c-488f-a6f6-1680b172b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:41.718705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98aca2-be4c-488f-a6f6-1680b172b6cc None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:41.719422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98aca2-be4c-488f-a6f6-1680b172b6cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:41.719722 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98aca2-be4c-488f-a6f6-1680b172b6cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:41.721588 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98aca2-be4c-488f-a6f6-1680b172b6cc None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:41.734946 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c30030-70c8-4277-93c5-db6e7d8f3a21 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:41.739684 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 403/803] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:41.742030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98aca2-be4c-488f-a6f6-1680b172b6cc None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:41.788173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee7c6-b31b-4b22-8aac-61873a5a1bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:41.789209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee7c6-b31b-4b22-8aac-61873a5a1bef None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:41.789707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee7c6-b31b-4b22-8aac-61873a5a1bef None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:41.790334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee7c6-b31b-4b22-8aac-61873a5a1bef None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:41.792308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee7c6-b31b-4b22-8aac-61873a5a1bef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:41.792851 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee7c6-b31b-4b22-8aac-61873a5a1bef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:41.794545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee7c6-b31b-4b22-8aac-61873a5a1bef None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:41.815234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 401/804] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:41.822720 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee7c6-b31b-4b22-8aac-61873a5a1bef None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:41.874472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98aca2-be4c-488f-a6f6-1680b172b6cc None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:41.910573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54adb5f4-1e11-4215-b009-2aed92ce934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:41.911561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adb5f4-1e11-4215-b009-2aed92ce934f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:41.911851 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adb5f4-1e11-4215-b009-2aed92ce934f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:41.912171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adb5f4-1e11-4215-b009-2aed92ce934f None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:41.913218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54adb5f4-1e11-4215-b009-2aed92ce934f None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:41.913526 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54adb5f4-1e11-4215-b009-2aed92ce934f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:41.915156 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54adb5f4-1e11-4215-b009-2aed92ce934f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:41.932065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 404/805] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:41.933227 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54adb5f4-1e11-4215-b009-2aed92ce934f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:41.947122 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee7c6-b31b-4b22-8aac-61873a5a1bef None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:41.982628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef97d6c-9ad5-4283-95e9-1ae6429b07d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:41.983465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef97d6c-9ad5-4283-95e9-1ae6429b07d8 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:41.983742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef97d6c-9ad5-4283-95e9-1ae6429b07d8 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:41.984042 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef97d6c-9ad5-4283-95e9-1ae6429b07d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:41.991525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef97d6c-9ad5-4283-95e9-1ae6429b07d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:41.992976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef97d6c-9ad5-4283-95e9-1ae6429b07d8 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-fef97d6c-9ad5-4283-95e9-1ae6429b07d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:42.007145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:42.008189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:42.009121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:42.009121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:42.009121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:42.009121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:42.009121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:42.009121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:42.009121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:42.009121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:42.009121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 04:13:42.009121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:42.009121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 402/806] 10.222.0.44 () {56 vars in 1117 bytes} [Mon May 9 04:13:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:42.009121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef97d6c-9ad5-4283-95e9-1ae6429b07d8 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:42.042304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54adb5f4-1e11-4215-b009-2aed92ce934f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:42.075160 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed13d3-43f8-423b-b742-2878d70dd5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:42.076194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed13d3-43f8-423b-b742-2878d70dd5de None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.076475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed13d3-43f8-423b-b742-2878d70dd5de None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.076748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed13d3-43f8-423b-b742-2878d70dd5de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.077577 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed13d3-43f8-423b-b742-2878d70dd5de None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=faf6107735424ff7901614953e7df403) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:42.077889 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed13d3-43f8-423b-b742-2878d70dd5de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:42.079561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed13d3-43f8-423b-b742-2878d70dd5de None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:42.085330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 405/807] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:41 2022] GET /identity/v3/role_assignments?user.id=faf6107735424ff7901614953e7df403&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=e59113c0366049e889df3f55eed90c72 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:42.086583 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706eb26a-9ddd-4828-a9ec-8a655d391cf8 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.086868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706eb26a-9ddd-4828-a9ec-8a655d391cf8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.087144 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706eb26a-9ddd-4828-a9ec-8a655d391cf8 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.088375 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 406/808] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:42.089061 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706eb26a-9ddd-4828-a9ec-8a655d391cf8 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:42.135625 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef97d6c-9ad5-4283-95e9-1ae6429b07d8 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:42.183325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c643b-0bb8-4d20-844a-73aea93d8c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:42.184224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c643b-0bb8-4d20-844a-73aea93d8c74 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.184703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c643b-0bb8-4d20-844a-73aea93d8c74 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.185008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c643b-0bb8-4d20-844a-73aea93d8c74 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.199044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c643b-0bb8-4d20-844a-73aea93d8c74 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:42.202266 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c643b-0bb8-4d20-844a-73aea93d8c74 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:42.219121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706eb26a-9ddd-4828-a9ec-8a655d391cf8 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-e06c643b-0bb8-4d20-844a-73aea93d8c74 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:42.224917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:42.225949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:42.226927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:42.226927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:42.226927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:42.226927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:42.226927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:42.226927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:42.226927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:42.226927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:42.226927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 04:13:42.226927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:42.235473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 403/809] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:13:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:42.235473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200ba0e-5875-409e-b2c6-8c16eb50ee92 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.235473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200ba0e-5875-409e-b2c6-8c16eb50ee92 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.235473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200ba0e-5875-409e-b2c6-8c16eb50ee92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.259241 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-a200ba0e-5875-409e-b2c6-8c16eb50ee92 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:42.260513 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a200ba0e-5875-409e-b2c6-8c16eb50ee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:42.276411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aaf6a9-d7d6-4235-95a0-7e842db2ee68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:42.277678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aaf6a9-d7d6-4235-95a0-7e842db2ee68 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.278293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aaf6a9-d7d6-4235-95a0-7e842db2ee68 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.279488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aaf6a9-d7d6-4235-95a0-7e842db2ee68 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.280609 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aaf6a9-d7d6-4235-95a0-7e842db2ee68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:42.282854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aaf6a9-d7d6-4235-95a0-7e842db2ee68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:42.284820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aaf6a9-d7d6-4235-95a0-7e842db2ee68 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:42.305401 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 407/810] 10.222.0.44 () {56 vars in 1149 bytes} [Mon May 9 04:13:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:42.306317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aaf6a9-d7d6-4235-95a0-7e842db2ee68 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:42.340877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 404/811] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:42.341600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200ba0e-5875-409e-b2c6-8c16eb50ee92 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:42.421261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aaf6a9-d7d6-4235-95a0-7e842db2ee68 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:42.454506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cbda9-ce6b-4d16-8a54-8b7695af2a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:42.455106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cbda9-ce6b-4d16-8a54-8b7695af2a86 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.455392 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cbda9-ce6b-4d16-8a54-8b7695af2a86 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.455678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cbda9-ce6b-4d16-8a54-8b7695af2a86 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.456546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cbda9-ce6b-4d16-8a54-8b7695af2a86 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:42.456856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cbda9-ce6b-4d16-8a54-8b7695af2a86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:42.458271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cbda9-ce6b-4d16-8a54-8b7695af2a86 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:42.463224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200ba0e-5875-409e-b2c6-8c16eb50ee92 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:42.480497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 408/812] 10.222.0.44 () {56 vars in 1135 bytes} [Mon May 9 04:13:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:42.480497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cbda9-ce6b-4d16-8a54-8b7695af2a86 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:42.502292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72361a-5125-46d8-8eab-7cd8da7422c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:42.503095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72361a-5125-46d8-8eab-7cd8da7422c5 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.503378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72361a-5125-46d8-8eab-7cd8da7422c5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.503650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72361a-5125-46d8-8eab-7cd8da7422c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.514280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72361a-5125-46d8-8eab-7cd8da7422c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:42.515742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72361a-5125-46d8-8eab-7cd8da7422c5 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-9c72361a-5125-46d8-8eab-7cd8da7422c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:42.534003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:42.535121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:42.536120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:42.536120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:42.536120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:42.536120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:42.536120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 405/813] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:42.536982 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac39669-187f-4588-8ef7-2f40cac3f200 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.537216 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac39669-187f-4588-8ef7-2f40cac3f200 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.537491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac39669-187f-4588-8ef7-2f40cac3f200 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.542901 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 406/814] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:42.542901 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac39669-187f-4588-8ef7-2f40cac3f200 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:42.585375 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cbda9-ce6b-4d16-8a54-8b7695af2a86 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:42.620368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29246697-307e-45ee-aaac-c8c8446686cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:42.621180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29246697-307e-45ee-aaac-c8c8446686cf None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.621466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29246697-307e-45ee-aaac-c8c8446686cf None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.621740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29246697-307e-45ee-aaac-c8c8446686cf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.622931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29246697-307e-45ee-aaac-c8c8446686cf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:42.624344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29246697-307e-45ee-aaac-c8c8446686cf None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-29246697-307e-45ee-aaac-c8c8446686cf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:42.636023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:42.637132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:42.638046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 409/815] 10.222.0.44 () {56 vars in 1129 bytes} [Mon May 9 04:13:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:42.638895 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29246697-307e-45ee-aaac-c8c8446686cf None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:42.649344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac39669-187f-4588-8ef7-2f40cac3f200 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:42.683081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018e69d-55b2-4d9f-bdf7-2c63bd017576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:42.683966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018e69d-55b2-4d9f-bdf7-2c63bd017576 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.684253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018e69d-55b2-4d9f-bdf7-2c63bd017576 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.684533 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018e69d-55b2-4d9f-bdf7-2c63bd017576 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.695211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018e69d-55b2-4d9f-bdf7-2c63bd017576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:42.696692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018e69d-55b2-4d9f-bdf7-2c63bd017576 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-a018e69d-55b2-4d9f-bdf7-2c63bd017576 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:42.717317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:42.718238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:42.719130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:42.719130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:42.719130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:42.719130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:42.719130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 407/816] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:42.719847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25504e-9097-4a64-b273-219e757a103c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.720043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25504e-9097-4a64-b273-219e757a103c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.720292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25504e-9097-4a64-b273-219e757a103c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.736198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-3c25504e-9097-4a64-b273-219e757a103c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:42.737355 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3c25504e-9097-4a64-b273-219e757a103c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:42.742399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29246697-307e-45ee-aaac-c8c8446686cf None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:42.779368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee289b4-0012-47c6-8748-bb5848b6959d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:42.780342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee289b4-0012-47c6-8748-bb5848b6959d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.780776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee289b4-0012-47c6-8748-bb5848b6959d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.781268 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee289b4-0012-47c6-8748-bb5848b6959d None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.782131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee289b4-0012-47c6-8748-bb5848b6959d None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:42.782584 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee289b4-0012-47c6-8748-bb5848b6959d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:42.784149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee289b4-0012-47c6-8748-bb5848b6959d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:42.796599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 410/817] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:42.797683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee289b4-0012-47c6-8748-bb5848b6959d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:42.812158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 408/818] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:42.819693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25504e-9097-4a64-b273-219e757a103c None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:42.895420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee289b4-0012-47c6-8748-bb5848b6959d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:42.920101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25504e-9097-4a64-b273-219e757a103c None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:42.927848 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37e56a-3fd6-4daa-813d-a19d2d9bbecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:42.928542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e56a-3fd6-4daa-813d-a19d2d9bbecc None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.928824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e56a-3fd6-4daa-813d-a19d2d9bbecc None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.929097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e56a-3fd6-4daa-813d-a19d2d9bbecc None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.929803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37e56a-3fd6-4daa-813d-a19d2d9bbecc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:42.930105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37e56a-3fd6-4daa-813d-a19d2d9bbecc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:42.931448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37e56a-3fd6-4daa-813d-a19d2d9bbecc None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:42.941165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 411/819] 10.222.0.44 () {56 vars in 1124 bytes} [Mon May 9 04:13:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:42.941165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37e56a-3fd6-4daa-813d-a19d2d9bbecc None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:42.959741 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a6c3a-d79d-4b3d-af28-faa461054476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:42.960438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a6c3a-d79d-4b3d-af28-faa461054476 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:42.960714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a6c3a-d79d-4b3d-af28-faa461054476 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:42.960985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a6c3a-d79d-4b3d-af28-faa461054476 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:42.961703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a6c3a-d79d-4b3d-af28-faa461054476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:42.962000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a6c3a-d79d-4b3d-af28-faa461054476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:42.963348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a6c3a-d79d-4b3d-af28-faa461054476 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:42.980051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 409/820] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:42.980051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a6c3a-d79d-4b3d-af28-faa461054476 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:43.042347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37e56a-3fd6-4daa-813d-a19d2d9bbecc None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:43.076779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78025a-6a06-4242-84ed-e571f1d284d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:43.077617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78025a-6a06-4242-84ed-e571f1d284d1 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:43.077902 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78025a-6a06-4242-84ed-e571f1d284d1 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:43.078181 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78025a-6a06-4242-84ed-e571f1d284d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:43.081618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a6c3a-d79d-4b3d-af28-faa461054476 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:43.084911 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78025a-6a06-4242-84ed-e571f1d284d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:43.086334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78025a-6a06-4242-84ed-e571f1d284d1 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-8a78025a-6a06-4242-84ed-e571f1d284d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:43.100355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:43.101466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:43.102587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:43.102587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:43.102587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:43.102587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:43.102587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:43.102587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:43.102587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:43.102587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:43.105743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 412/821] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:43.105743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78025a-6a06-4242-84ed-e571f1d284d1 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:43.117859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc564d-cded-403a-8bf4-c5ac43fbaa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:43.118671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc564d-cded-403a-8bf4-c5ac43fbaa0a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:43.118952 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc564d-cded-403a-8bf4-c5ac43fbaa0a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:43.119223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc564d-cded-403a-8bf4-c5ac43fbaa0a None admin] PATH_INFO: `/v3/services/network` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:43.119867 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc564d-cded-403a-8bf4-c5ac43fbaa0a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:43.121247 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc564d-cded-403a-8bf4-c5ac43fbaa0a None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-eefc564d-cded-403a-8bf4-c5ac43fbaa0a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:43.133357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:43.134560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 04:13:43.135728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:43.139395 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 410/822] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:42 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:43.140078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc564d-cded-403a-8bf4-c5ac43fbaa0a None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:43.205854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78025a-6a06-4242-84ed-e571f1d284d1 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:43.233789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc564d-cded-403a-8bf4-c5ac43fbaa0a None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:43.238053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02203dc3-be52-4232-bfa5-70c9d18cef22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:43.238715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02203dc3-be52-4232-bfa5-70c9d18cef22 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:43.239002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02203dc3-be52-4232-bfa5-70c9d18cef22 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:43.239276 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02203dc3-be52-4232-bfa5-70c9d18cef22 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:43.240007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02203dc3-be52-4232-bfa5-70c9d18cef22 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:43.240308 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02203dc3-be52-4232-bfa5-70c9d18cef22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:43.241657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02203dc3-be52-4232-bfa5-70c9d18cef22 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:43.251439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 413/823] 10.222.0.44 () {56 vars in 1124 bytes} [Mon May 9 04:13:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:43.251439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02203dc3-be52-4232-bfa5-70c9d18cef22 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:43.273055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d7913-23f2-4045-a561-2b5eb2dcf9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:43.273850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d7913-23f2-4045-a561-2b5eb2dcf9eb None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:43.274133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d7913-23f2-4045-a561-2b5eb2dcf9eb None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:43.274406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d7913-23f2-4045-a561-2b5eb2dcf9eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:43.281186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d7913-23f2-4045-a561-2b5eb2dcf9eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:43.282721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d7913-23f2-4045-a561-2b5eb2dcf9eb None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-ff8d7913-23f2-4045-a561-2b5eb2dcf9eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:43.298086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:43.299024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:43.300371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:43.300371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:43.300371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:43.300371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:43.300371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:43.300371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:43.300371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:43.300371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:43.304138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 411/824] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:43.304957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d7913-23f2-4045-a561-2b5eb2dcf9eb None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:43.352398 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02203dc3-be52-4232-bfa5-70c9d18cef22 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:43.385566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3fe02-b2b2-4e5d-8758-2337203ac47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:43.386220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3fe02-b2b2-4e5d-8758-2337203ac47c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:43.386499 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3fe02-b2b2-4e5d-8758-2337203ac47c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:43.386768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3fe02-b2b2-4e5d-8758-2337203ac47c None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:43.387537 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3fe02-b2b2-4e5d-8758-2337203ac47c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:43.387869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3fe02-b2b2-4e5d-8758-2337203ac47c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:43.389303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3fe02-b2b2-4e5d-8758-2337203ac47c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:43.400730 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 414/825] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:43.401495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3fe02-b2b2-4e5d-8758-2337203ac47c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:43.406921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d7913-23f2-4045-a561-2b5eb2dcf9eb None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:43.440660 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b1ba1-57d1-41d1-a015-0cc2ea241192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:43.441328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b1ba1-57d1-41d1-a015-0cc2ea241192 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:43.441608 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b1ba1-57d1-41d1-a015-0cc2ea241192 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:43.441882 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b1ba1-57d1-41d1-a015-0cc2ea241192 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:43.442583 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b1ba1-57d1-41d1-a015-0cc2ea241192 None admin] RBAC: Adding query filter params (name=network) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:43.443124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b1ba1-57d1-41d1-a015-0cc2ea241192 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:43.444274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b1ba1-57d1-41d1-a015-0cc2ea241192 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:43.448705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 412/826] 10.222.0.44 () {56 vars in 1112 bytes} [Mon May 9 04:13:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:43.449542 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b1ba1-57d1-41d1-a015-0cc2ea241192 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:43.499775 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3fe02-b2b2-4e5d-8758-2337203ac47c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:43.532371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e04bb-5072-4b00-b3c6-0fc33f18739e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:43.533032 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e04bb-5072-4b00-b3c6-0fc33f18739e None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:43.533317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e04bb-5072-4b00-b3c6-0fc33f18739e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:43.533591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e04bb-5072-4b00-b3c6-0fc33f18739e None admin] PATH_INFO: `/v3/endpoints` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:43.534323 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e04bb-5072-4b00-b3c6-0fc33f18739e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:43.535679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e04bb-5072-4b00-b3c6-0fc33f18739e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:43.549240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b1ba1-57d1-41d1-a015-0cc2ea241192 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:43.555808 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 415/827] 10.222.0.44 () {60 vars in 1144 bytes} [Mon May 9 04:13:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:13:43.565037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e04bb-5072-4b00-b3c6-0fc33f18739e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:43.596106 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c5b56-3b21-4dce-8795-9eeff0735894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:43.596933 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c5b56-3b21-4dce-8795-9eeff0735894 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:43.597216 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c5b56-3b21-4dce-8795-9eeff0735894 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:43.597492 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c5b56-3b21-4dce-8795-9eeff0735894 None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:43.598473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c5b56-3b21-4dce-8795-9eeff0735894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:43.598793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c5b56-3b21-4dce-8795-9eeff0735894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:43.600995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c5b56-3b21-4dce-8795-9eeff0735894 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:43.615588 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 413/828] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:43.616494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c5b56-3b21-4dce-8795-9eeff0735894 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:43.679486 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e04bb-5072-4b00-b3c6-0fc33f18739e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:43.717989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe9352-2e07-42a8-a1c6-f06f3dfefce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:43.719357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe9352-2e07-42a8-a1c6-f06f3dfefce1 None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:43.719837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe9352-2e07-42a8-a1c6-f06f3dfefce1 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:43.720371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe9352-2e07-42a8-a1c6-f06f3dfefce1 None admin] PATH_INFO: `/v3/projects/3322b00723974bd99e837d447bac22d1/users/ee0055c75b91471a99d9bb18248839ea/roles/c255c186230d4339b46d5e1f7f1f6cbc` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:43.721644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c5b56-3b21-4dce-8795-9eeff0735894 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:43.737190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe9352-2e07-42a8-a1c6-f06f3dfefce1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3322b00723974bd99e837d447bac22d1, user_id=ee0055c75b91471a99d9bb18248839ea, role_id=c255c186230d4339b46d5e1f7f1f6cbc)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:43.739017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe9352-2e07-42a8-a1c6-f06f3dfefce1 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:43.768425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 416/829] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:13:43 2022] PUT /identity/v3/projects/3322b00723974bd99e837d447bac22d1/users/ee0055c75b91471a99d9bb18248839ea/roles/c255c186230d4339b46d5e1f7f1f6cbc => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:13:43.769472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe9352-2e07-42a8-a1c6-f06f3dfefce1 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:43.770380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b2cdb-0f05-48c1-a5e1-ac333f51f93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:43.771291 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b2cdb-0f05-48c1-a5e1-ac333f51f93d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:43.771784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b2cdb-0f05-48c1-a5e1-ac333f51f93d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:43.772529 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b2cdb-0f05-48c1-a5e1-ac333f51f93d None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:43.774448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b2cdb-0f05-48c1-a5e1-ac333f51f93d None admin] RBAC: Adding query filter params (type=network) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:43.775004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b2cdb-0f05-48c1-a5e1-ac333f51f93d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:43.776754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b2cdb-0f05-48c1-a5e1-ac333f51f93d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:43.785479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 414/830] 10.222.0.44 () {56 vars in 1112 bytes} [Mon May 9 04:13:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:43.791087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b2cdb-0f05-48c1-a5e1-ac333f51f93d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:43.888891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe9352-2e07-42a8-a1c6-f06f3dfefce1 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:43.942536 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b2cdb-0f05-48c1-a5e1-ac333f51f93d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:43.948283 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65efc58-6ab4-448e-b4fc-bb4330a0b736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:43.949244 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65efc58-6ab4-448e-b4fc-bb4330a0b736 None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:43.949526 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65efc58-6ab4-448e-b4fc-bb4330a0b736 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:43.949798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65efc58-6ab4-448e-b4fc-bb4330a0b736 None admin] PATH_INFO: `/v3/projects/3322b00723974bd99e837d447bac22d1/users/c46e7cc6407f44f08a659288e3ad0767/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:43.957253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65efc58-6ab4-448e-b4fc-bb4330a0b736 None admin] RBAC: Authorizing `identity:create_grant(project_id=3322b00723974bd99e837d447bac22d1, user_id=c46e7cc6407f44f08a659288e3ad0767, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:43.958819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65efc58-6ab4-448e-b4fc-bb4330a0b736 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:43.985008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 417/831] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:13:43 2022] PUT /identity/v3/projects/3322b00723974bd99e837d447bac22d1/users/c46e7cc6407f44f08a659288e3ad0767/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:13:43.997117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf62262-442c-4479-876c-0517599cca66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:43.998558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf62262-442c-4479-876c-0517599cca66 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:43.998905 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf62262-442c-4479-876c-0517599cca66 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:43.999178 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf62262-442c-4479-876c-0517599cca66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:44.000012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf62262-442c-4479-876c-0517599cca66 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:44.001540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf62262-442c-4479-876c-0517599cca66 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:44.033380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 415/832] 10.222.0.44 () {60 vars in 1144 bytes} [Mon May 9 04:13:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:13:44.509268 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0afe9e-fc80-49dc-8ee0-52d677ef7995 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:44.509781 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0afe9e-fc80-49dc-8ee0-52d677ef7995 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:44.509888 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0afe9e-fc80-49dc-8ee0-52d677ef7995 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:44.511413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 418/833] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:44.517704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11562f-63df-4dee-bc52-23ff66e060df None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:44.518003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11562f-63df-4dee-bc52-23ff66e060df None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:44.518286 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11562f-63df-4dee-bc52-23ff66e060df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:44.535567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-5e11562f-63df-4dee-bc52-23ff66e060df None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:44.536770 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5e11562f-63df-4dee-bc52-23ff66e060df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:44.615680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 416/834] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:44.638506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0afe9e-fc80-49dc-8ee0-52d677ef7995 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:44.744150 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0afe9e-fc80-49dc-8ee0-52d677ef7995 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:44.783917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9792494-0cb0-44f6-a977-7d61baa78b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:44.784667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9792494-0cb0-44f6-a977-7d61baa78b87 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:44.784953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9792494-0cb0-44f6-a977-7d61baa78b87 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:44.785225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9792494-0cb0-44f6-a977-7d61baa78b87 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:44.785883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9792494-0cb0-44f6-a977-7d61baa78b87 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:44.787242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9792494-0cb0-44f6-a977-7d61baa78b87 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-f9792494-0cb0-44f6-a977-7d61baa78b87 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:44.800099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:44.801081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:44.801913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 419/835] 10.222.0.44 () {56 vars in 1132 bytes} [Mon May 9 04:13:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:44.807521 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11562f-63df-4dee-bc52-23ff66e060df None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:44.923829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11562f-63df-4dee-bc52-23ff66e060df None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:44.962576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26868d9-3310-4bac-b138-e0ae17ac05d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:44.963244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26868d9-3310-4bac-b138-e0ae17ac05d1 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:44.963523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26868d9-3310-4bac-b138-e0ae17ac05d1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:44.963798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26868d9-3310-4bac-b138-e0ae17ac05d1 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:44.964531 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26868d9-3310-4bac-b138-e0ae17ac05d1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:44.964831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26868d9-3310-4bac-b138-e0ae17ac05d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:44.968017 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26868d9-3310-4bac-b138-e0ae17ac05d1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:44.973328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 417/836] 10.222.0.44 () {56 vars in 1126 bytes} [Mon May 9 04:13:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:44.978896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9792494-0cb0-44f6-a977-7d61baa78b87 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:45.103462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9792494-0cb0-44f6-a977-7d61baa78b87 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:45.141664 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b4cef-741c-4bbe-89e3-6fc6525d6178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:45.142567 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b4cef-741c-4bbe-89e3-6fc6525d6178 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:45.142969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b4cef-741c-4bbe-89e3-6fc6525d6178 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:45.143270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b4cef-741c-4bbe-89e3-6fc6525d6178 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:45.144067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b4cef-741c-4bbe-89e3-6fc6525d6178 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:45.144616 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b4cef-741c-4bbe-89e3-6fc6525d6178 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:45.146439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b4cef-741c-4bbe-89e3-6fc6525d6178 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:45.151595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 420/837] 10.222.0.44 () {56 vars in 1126 bytes} [Mon May 9 04:13:44 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:46.070444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b643a-2058-4abe-aac3-fdc69c614894 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:46.071114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b643a-2058-4abe-aac3-fdc69c614894 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:46.071114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b643a-2058-4abe-aac3-fdc69c614894 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:46.072566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 418/838] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:46.078321 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26418b3-220c-42b9-a20a-584e164735ce None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:46.079057 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26418b3-220c-42b9-a20a-584e164735ce None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:46.079212 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26418b3-220c-42b9-a20a-584e164735ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:46.095633 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d26418b3-220c-42b9-a20a-584e164735ce None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:46.096836 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d26418b3-220c-42b9-a20a-584e164735ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:46.168006 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 421/839] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:46.184432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b643a-2058-4abe-aac3-fdc69c614894 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:46.292616 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b643a-2058-4abe-aac3-fdc69c614894 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:46.330107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007fd60-1003-4a47-b43a-700330ccc0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:46.330915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007fd60-1003-4a47-b43a-700330ccc0da None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:46.331195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007fd60-1003-4a47-b43a-700330ccc0da None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:46.331462 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007fd60-1003-4a47-b43a-700330ccc0da None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:46.332161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007fd60-1003-4a47-b43a-700330ccc0da None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:46.333578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007fd60-1003-4a47-b43a-700330ccc0da None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-a007fd60-1003-4a47-b43a-700330ccc0da None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:46.347163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:46.349543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 04:13:46.351530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:46.357447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 419/840] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:13:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:46.357599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26418b3-220c-42b9-a20a-584e164735ce None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:46.406093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b904901-fe4d-4397-82b7-ff2dffbf785c None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:46.406425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b904901-fe4d-4397-82b7-ff2dffbf785c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:46.406647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b904901-fe4d-4397-82b7-ff2dffbf785c None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:46.408159 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 420/841] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:13:46.414270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a8f47-33c2-4462-a3fd-2ab8c79b7995 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:46.414816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a8f47-33c2-4462-a3fd-2ab8c79b7995 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:46.415301 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a8f47-33c2-4462-a3fd-2ab8c79b7995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:46.432910 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-b53a8f47-33c2-4462-a3fd-2ab8c79b7995 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:46.434475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b53a8f47-33c2-4462-a3fd-2ab8c79b7995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:46.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26418b3-220c-42b9-a20a-584e164735ce None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:46.494432 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc6ffc-52d3-4962-ab6c-642baf0d91c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:46.495098 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc6ffc-52d3-4962-ab6c-642baf0d91c8 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:46.495381 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc6ffc-52d3-4962-ab6c-642baf0d91c8 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:46.495655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc6ffc-52d3-4962-ab6c-642baf0d91c8 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:46.496397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc6ffc-52d3-4962-ab6c-642baf0d91c8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:46.496702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc6ffc-52d3-4962-ab6c-642baf0d91c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:46.498195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc6ffc-52d3-4962-ab6c-642baf0d91c8 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:46.502478 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 422/842] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:13:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:46.503679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a018df-b3af-4fad-a2ee-a43447b947e8 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:46.503991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a018df-b3af-4fad-a2ee-a43447b947e8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:46.504270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a018df-b3af-4fad-a2ee-a43447b947e8 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:46.505604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 423/843] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:46.507808 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a018df-b3af-4fad-a2ee-a43447b947e8 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:46.515318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 421/844] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:13:46.515318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33f1a0-3211-4279-8831-6e747323551c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:46.515318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33f1a0-3211-4279-8831-6e747323551c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:46.515318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33f1a0-3211-4279-8831-6e747323551c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:46.534091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-3f33f1a0-3211-4279-8831-6e747323551c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:46.535939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3f33f1a0-3211-4279-8831-6e747323551c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:46.602974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 422/845] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:46.603535 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33f1a0-3211-4279-8831-6e747323551c None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:46.613013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a018df-b3af-4fad-a2ee-a43447b947e8 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:46.646376 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793a6c2-ce12-43b2-81c1-a246ee845f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:46.647042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793a6c2-ce12-43b2-81c1-a246ee845f0f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:46.647322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793a6c2-ce12-43b2-81c1-a246ee845f0f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:46.647598 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793a6c2-ce12-43b2-81c1-a246ee845f0f None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:46.648342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793a6c2-ce12-43b2-81c1-a246ee845f0f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:46.648645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793a6c2-ce12-43b2-81c1-a246ee845f0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:46.649995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793a6c2-ce12-43b2-81c1-a246ee845f0f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:46.663921 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 424/846] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:13:46 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:46.663921 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793a6c2-ce12-43b2-81c1-a246ee845f0f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:46.737788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33f1a0-3211-4279-8831-6e747323551c None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:46.785273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d5011-f7e2-4ab7-8b67-654fa2744791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:46.786090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d5011-f7e2-4ab7-8b67-654fa2744791 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:46.786369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d5011-f7e2-4ab7-8b67-654fa2744791 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:46.786642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d5011-f7e2-4ab7-8b67-654fa2744791 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:46.789469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793a6c2-ce12-43b2-81c1-a246ee845f0f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:46.802433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d5011-f7e2-4ab7-8b67-654fa2744791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-f57d5011-f7e2-4ab7-8b67-654fa2744791 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:46.813416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:46.814429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:46.815283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:46.815283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:46.815283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:46.815283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:46.815283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:46.815283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:46.815283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:46.815283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:46.815283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:46.815283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:46.815283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:13:46.815283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:46.820039 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 423/847] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:13:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:46.824255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d5011-f7e2-4ab7-8b67-654fa2744791 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:46.837653 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b0b3d-0d3a-4383-9506-b6b5e1c69877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:46.838855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b0b3d-0d3a-4383-9506-b6b5e1c69877 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:46.839208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b0b3d-0d3a-4383-9506-b6b5e1c69877 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:46.839559 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b0b3d-0d3a-4383-9506-b6b5e1c69877 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:46.856021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b0b3d-0d3a-4383-9506-b6b5e1c69877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-487b0b3d-0d3a-4383-9506-b6b5e1c69877 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:46.865734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:46.866909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:46.867966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:46.867966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:46.867966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:46.867966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:46.867966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:46.867966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:46.867966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:46.867966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:46.867966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:46.867966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:46.867966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:13:46.867966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:46.870405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 425/848] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:13:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:46.878196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b0b3d-0d3a-4383-9506-b6b5e1c69877 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:46.938476 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d5011-f7e2-4ab7-8b67-654fa2744791 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:46.973958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f56757-f69f-4416-813c-7530c5f4ed96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:46.974654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f56757-f69f-4416-813c-7530c5f4ed96 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:46.974936 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f56757-f69f-4416-813c-7530c5f4ed96 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:46.975207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f56757-f69f-4416-813c-7530c5f4ed96 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:46.975981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f56757-f69f-4416-813c-7530c5f4ed96 None admin] RBAC: Adding query filter params (name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:46.976284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f56757-f69f-4416-813c-7530c5f4ed96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:46.977649 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f56757-f69f-4416-813c-7530c5f4ed96 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:46.989873 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b0b3d-0d3a-4383-9506-b6b5e1c69877 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:46.990480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 424/849] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:13:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:46.991692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f56757-f69f-4416-813c-7530c5f4ed96 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:47.024308 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0297d5-01d2-470a-adfc-fbf09b3bf53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:47.025033 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0297d5-01d2-470a-adfc-fbf09b3bf53b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:47.025321 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0297d5-01d2-470a-adfc-fbf09b3bf53b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:47.025600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0297d5-01d2-470a-adfc-fbf09b3bf53b None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:47.026349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0297d5-01d2-470a-adfc-fbf09b3bf53b None admin] RBAC: Adding query filter params (name=admin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:47.026654 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0297d5-01d2-470a-adfc-fbf09b3bf53b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:47.028043 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0297d5-01d2-470a-adfc-fbf09b3bf53b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:47.038026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 426/850] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:13:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:47.043391 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0297d5-01d2-470a-adfc-fbf09b3bf53b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:47.093107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f56757-f69f-4416-813c-7530c5f4ed96 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:47.125886 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af7fae-86cd-4627-a739-b132274e59ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:47.126609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af7fae-86cd-4627-a739-b132274e59ea None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:47.126887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af7fae-86cd-4627-a739-b132274e59ea None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:47.127155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af7fae-86cd-4627-a739-b132274e59ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:47.138477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af7fae-86cd-4627-a739-b132274e59ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:47.139929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af7fae-86cd-4627-a739-b132274e59ea None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:47.144788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0297d5-01d2-470a-adfc-fbf09b3bf53b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-d7af7fae-86cd-4627-a739-b132274e59ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:47.157345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:47.159359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:47.161469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:47.161469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:47.161469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:47.161469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:47.163923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 425/851] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:47.164789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af7fae-86cd-4627-a739-b132274e59ea None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:47.179791 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b305ca-8b00-410b-b02a-2d52f9fb0028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:47.180623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b305ca-8b00-410b-b02a-2d52f9fb0028 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:47.180905 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b305ca-8b00-410b-b02a-2d52f9fb0028 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:47.181177 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b305ca-8b00-410b-b02a-2d52f9fb0028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:47.193117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b305ca-8b00-410b-b02a-2d52f9fb0028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:47.194566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b305ca-8b00-410b-b02a-2d52f9fb0028 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-93b305ca-8b00-410b-b02a-2d52f9fb0028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:47.212910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:47.214955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:47.217904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:47.217904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:47.217904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:47.217904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:47.217904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 427/852] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:47.223916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b305ca-8b00-410b-b02a-2d52f9fb0028 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:47.266692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af7fae-86cd-4627-a739-b132274e59ea None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:47.303641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85612579-236e-4864-a243-0bf64699302b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:47.304361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85612579-236e-4864-a243-0bf64699302b None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:47.304638 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85612579-236e-4864-a243-0bf64699302b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:47.304910 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85612579-236e-4864-a243-0bf64699302b None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:47.305655 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85612579-236e-4864-a243-0bf64699302b None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:47.305956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85612579-236e-4864-a243-0bf64699302b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:47.307360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85612579-236e-4864-a243-0bf64699302b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:47.324437 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 426/853] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:47.325690 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b305ca-8b00-410b-b02a-2d52f9fb0028 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:47.325877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85612579-236e-4864-a243-0bf64699302b None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:47.360633 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bbd56-ee35-4903-93c4-c2338e755868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:47.361301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bbd56-ee35-4903-93c4-c2338e755868 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:47.361627 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bbd56-ee35-4903-93c4-c2338e755868 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:47.361910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bbd56-ee35-4903-93c4-c2338e755868 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:47.362629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bbd56-ee35-4903-93c4-c2338e755868 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:47.362935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bbd56-ee35-4903-93c4-c2338e755868 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:47.364318 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bbd56-ee35-4903-93c4-c2338e755868 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:47.381511 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 428/854] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:47.382492 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bbd56-ee35-4903-93c4-c2338e755868 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:47.424218 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85612579-236e-4864-a243-0bf64699302b None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:47.458088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89195d7c-c26b-4306-b76b-46d0891f2bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:47.458911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89195d7c-c26b-4306-b76b-46d0891f2bfb None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:47.459182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89195d7c-c26b-4306-b76b-46d0891f2bfb None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:47.459450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89195d7c-c26b-4306-b76b-46d0891f2bfb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:47.467354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89195d7c-c26b-4306-b76b-46d0891f2bfb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:47.468823 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89195d7c-c26b-4306-b76b-46d0891f2bfb None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-89195d7c-c26b-4306-b76b-46d0891f2bfb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:47.484963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:47.485986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:47.486938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:47.486938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:47.486938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:47.486938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:47.486938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:47.486938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:47.486938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:47.486938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:47.486938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 04:13:47.486938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:47.490022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bbd56-ee35-4903-93c4-c2338e755868 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:47.490668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 427/855] 10.222.0.44 () {56 vars in 1117 bytes} [Mon May 9 04:13:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:47.491978 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36f304-7ff3-49e3-97ad-c2600444092c None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:47.492266 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36f304-7ff3-49e3-97ad-c2600444092c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:47.492539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36f304-7ff3-49e3-97ad-c2600444092c None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:47.493923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 428/856] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:47.495930 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36f304-7ff3-49e3-97ad-c2600444092c None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:47.534062 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72983fa-f5d2-42ac-bb8d-04b6e42876b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:47.535264 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72983fa-f5d2-42ac-bb8d-04b6e42876b1 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:47.535716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72983fa-f5d2-42ac-bb8d-04b6e42876b1 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:47.536186 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72983fa-f5d2-42ac-bb8d-04b6e42876b1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:47.545931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72983fa-f5d2-42ac-bb8d-04b6e42876b1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:47.548291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72983fa-f5d2-42ac-bb8d-04b6e42876b1 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-b72983fa-f5d2-42ac-bb8d-04b6e42876b1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:47.565564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:47.566552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:47.572576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:47.572576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:47.572576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:47.572576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:47.572576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:47.572576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:47.572576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:47.572576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:47.572576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 04:13:47.572576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:47.572576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 429/857] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:13:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:47.572576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf9487-84b8-41a2-bcd1-8b2c0eabde1b None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:47.572576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf9487-84b8-41a2-bcd1-8b2c0eabde1b None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:47.573290 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf9487-84b8-41a2-bcd1-8b2c0eabde1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:47.588819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-3cbf9487-84b8-41a2-bcd1-8b2c0eabde1b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:47.590025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3cbf9487-84b8-41a2-bcd1-8b2c0eabde1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:47.601322 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36f304-7ff3-49e3-97ad-c2600444092c None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:47.632858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d39d3-8679-4556-aa38-a4e621ea95a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:47.633451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d39d3-8679-4556-aa38-a4e621ea95a1 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:47.633734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d39d3-8679-4556-aa38-a4e621ea95a1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:47.634002 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d39d3-8679-4556-aa38-a4e621ea95a1 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:47.634829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d39d3-8679-4556-aa38-a4e621ea95a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:47.635130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d39d3-8679-4556-aa38-a4e621ea95a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:47.636563 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d39d3-8679-4556-aa38-a4e621ea95a1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:47.658593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 429/858] 10.222.0.44 () {56 vars in 1149 bytes} [Mon May 9 04:13:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:47.658593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d39d3-8679-4556-aa38-a4e621ea95a1 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:47.660519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 430/859] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:47.668843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf9487-84b8-41a2-bcd1-8b2c0eabde1b None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:47.758377 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d39d3-8679-4556-aa38-a4e621ea95a1 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:47.779352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf9487-84b8-41a2-bcd1-8b2c0eabde1b None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:47.791842 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fceac5b-83e2-464f-bb6a-b3c363de2179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:47.792651 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fceac5b-83e2-464f-bb6a-b3c363de2179 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:47.792939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fceac5b-83e2-464f-bb6a-b3c363de2179 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:47.793214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fceac5b-83e2-464f-bb6a-b3c363de2179 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:47.794569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fceac5b-83e2-464f-bb6a-b3c363de2179 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:47.794876 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fceac5b-83e2-464f-bb6a-b3c363de2179 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:47.796512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fceac5b-83e2-464f-bb6a-b3c363de2179 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:47.813138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2009b-6deb-43ab-b8a2-e12eddabfa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:47.818105 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 430/860] 10.222.0.44 () {56 vars in 1135 bytes} [Mon May 9 04:13:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:47.818236 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2009b-6deb-43ab-b8a2-e12eddabfa47 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:47.818236 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2009b-6deb-43ab-b8a2-e12eddabfa47 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:47.818236 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2009b-6deb-43ab-b8a2-e12eddabfa47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:47.818473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fceac5b-83e2-464f-bb6a-b3c363de2179 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:47.885681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2009b-6deb-43ab-b8a2-e12eddabfa47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:47.887032 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2009b-6deb-43ab-b8a2-e12eddabfa47 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:47.923786 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fceac5b-83e2-464f-bb6a-b3c363de2179 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:47.961161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc66324-3b5f-48ef-bae4-f6a1439c686c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:47.961832 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc66324-3b5f-48ef-bae4-f6a1439c686c None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:47.962110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc66324-3b5f-48ef-bae4-f6a1439c686c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:47.962384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc66324-3b5f-48ef-bae4-f6a1439c686c None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:47.963092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc66324-3b5f-48ef-bae4-f6a1439c686c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:47.964480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc66324-3b5f-48ef-bae4-f6a1439c686c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:47.984067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 431/861] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:13:47 2022] POST /identity/v3/services => generated 269 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:13:47.984067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc66324-3b5f-48ef-bae4-f6a1439c686c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:47.989701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 431/862] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:47 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:13:47.995688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2009b-6deb-43ab-b8a2-e12eddabfa47 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:48.100125 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2009b-6deb-43ab-b8a2-e12eddabfa47 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:48.126504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc66324-3b5f-48ef-bae4-f6a1439c686c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:48.139268 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b1ecd-2539-4731-9351-173d391ac572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:48.140375 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b1ecd-2539-4731-9351-173d391ac572 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:48.140801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b1ecd-2539-4731-9351-173d391ac572 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:48.141214 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b1ecd-2539-4731-9351-173d391ac572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:48.152219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b1ecd-2539-4731-9351-173d391ac572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:48.154933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b1ecd-2539-4731-9351-173d391ac572 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:48.163455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f4334-c971-4842-87c0-03dd5729ab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:48.164201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f4334-c971-4842-87c0-03dd5729ab14 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:48.164480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f4334-c971-4842-87c0-03dd5729ab14 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:48.164746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f4334-c971-4842-87c0-03dd5729ab14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-450b1ecd-2539-4731-9351-173d391ac572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:48.175538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:48.176745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:48.177900 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:48.177900 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:48.177900 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:48.177900 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:48.179750 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 432/863] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:48.186339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b1ecd-2539-4731-9351-173d391ac572 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:48.252785 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f4334-c971-4842-87c0-03dd5729ab14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:48.254239 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f4334-c971-4842-87c0-03dd5729ab14 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:48.306478 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b1ecd-2539-4731-9351-173d391ac572 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:48.343706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdbcaf-fc45-4339-8610-85b88946cd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:48.344645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdbcaf-fc45-4339-8610-85b88946cd35 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:48.345116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdbcaf-fc45-4339-8610-85b88946cd35 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:48.345556 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdbcaf-fc45-4339-8610-85b88946cd35 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:48.346421 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdbcaf-fc45-4339-8610-85b88946cd35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:48.346876 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdbcaf-fc45-4339-8610-85b88946cd35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:48.348405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdbcaf-fc45-4339-8610-85b88946cd35 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:48.353434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 432/864] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:47 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:13:48.361848 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f4334-c971-4842-87c0-03dd5729ab14 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:48.364821 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 433/865] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:48.375792 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdbcaf-fc45-4339-8610-85b88946cd35 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:48.468499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f4334-c971-4842-87c0-03dd5729ab14 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:48.496687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdbcaf-fc45-4339-8610-85b88946cd35 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:48.508542 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fced0a-3698-4282-a44e-6f337872a782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:48.509679 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fced0a-3698-4282-a44e-6f337872a782 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:48.510161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fced0a-3698-4282-a44e-6f337872a782 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:48.510637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fced0a-3698-4282-a44e-6f337872a782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:48.522118 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fced0a-3698-4282-a44e-6f337872a782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:48.524291 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fced0a-3698-4282-a44e-6f337872a782 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:48.534214 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fef327-dc84-4901-9405-2933dd3c10c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:48.535337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fef327-dc84-4901-9405-2933dd3c10c6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:48.535818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fef327-dc84-4901-9405-2933dd3c10c6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:48.536370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fef327-dc84-4901-9405-2933dd3c10c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:48.543318 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fef327-dc84-4901-9405-2933dd3c10c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-18fced0a-3698-4282-a44e-6f337872a782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:48.545754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:48.546628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:48.547517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:48.547517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:48.547517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:48.547517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:48.548210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fef327-dc84-4901-9405-2933dd3c10c6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:48.551705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 433/866] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:48.553346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fced0a-3698-4282-a44e-6f337872a782 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-63fef327-dc84-4901-9405-2933dd3c10c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:48.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:48.570952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:48.571855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:48.571855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:48.571855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:48.571855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:48.571855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:48.571855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:48.571855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:48.571855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:48.578169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 434/867] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:48.579020 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fef327-dc84-4901-9405-2933dd3c10c6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:48.664247 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fced0a-3698-4282-a44e-6f337872a782 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:48.701020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58497c2-3c6d-4633-a4f0-a40426ac8e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:48.702046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58497c2-3c6d-4633-a4f0-a40426ac8e57 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:48.702532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58497c2-3c6d-4633-a4f0-a40426ac8e57 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:48.702636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fef327-dc84-4901-9405-2933dd3c10c6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:48.703155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58497c2-3c6d-4633-a4f0-a40426ac8e57 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:48.704357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58497c2-3c6d-4633-a4f0-a40426ac8e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:48.704881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58497c2-3c6d-4633-a4f0-a40426ac8e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:48.706456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58497c2-3c6d-4633-a4f0-a40426ac8e57 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:48.719329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 434/868] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:48.724716 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58497c2-3c6d-4633-a4f0-a40426ac8e57 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:48.750339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b722fc1-5c5d-4151-83fd-81af02802a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:48.751033 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b722fc1-5c5d-4151-83fd-81af02802a32 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:48.751325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b722fc1-5c5d-4151-83fd-81af02802a32 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:48.751602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b722fc1-5c5d-4151-83fd-81af02802a32 None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:48.752593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b722fc1-5c5d-4151-83fd-81af02802a32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:48.752905 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b722fc1-5c5d-4151-83fd-81af02802a32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:48.755001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b722fc1-5c5d-4151-83fd-81af02802a32 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:48.774726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 435/869] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:48.776474 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b722fc1-5c5d-4151-83fd-81af02802a32 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:48.843561 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58497c2-3c6d-4633-a4f0-a40426ac8e57 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:48.879219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65249e22-2374-4eb1-8e37-b6c545199590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:48.881756 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65249e22-2374-4eb1-8e37-b6c545199590 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:48.882291 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65249e22-2374-4eb1-8e37-b6c545199590 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:48.882802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65249e22-2374-4eb1-8e37-b6c545199590 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:48.890714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65249e22-2374-4eb1-8e37-b6c545199590 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:48.892496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65249e22-2374-4eb1-8e37-b6c545199590 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:48.899606 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b722fc1-5c5d-4151-83fd-81af02802a32 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-65249e22-2374-4eb1-8e37-b6c545199590 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:48.908063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:48.908956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:48.909847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:48.909847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:48.909847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:48.909847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:48.909847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:48.909847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:48.909847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:48.909847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:48.913238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 435/870] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:48.918821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65249e22-2374-4eb1-8e37-b6c545199590 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:48.939338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67a8b0-8554-4f6d-a8c7-51fe8db4ecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:48.940030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67a8b0-8554-4f6d-a8c7-51fe8db4ecf9 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:48.940320 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67a8b0-8554-4f6d-a8c7-51fe8db4ecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:48.940599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67a8b0-8554-4f6d-a8c7-51fe8db4ecf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:48.941429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67a8b0-8554-4f6d-a8c7-51fe8db4ecf9 None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=ee0055c75b91471a99d9bb18248839ea) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:48.941741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67a8b0-8554-4f6d-a8c7-51fe8db4ecf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:48.943168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67a8b0-8554-4f6d-a8c7-51fe8db4ecf9 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:48.955930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 436/871] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:48 2022] GET /identity/v3/role_assignments?user.id=ee0055c75b91471a99d9bb18248839ea&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:48.985933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55481218-e7f3-4997-8218-3a5a80a6ce14 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:48.986127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55481218-e7f3-4997-8218-3a5a80a6ce14 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:48.986410 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55481218-e7f3-4997-8218-3a5a80a6ce14 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:48.992365 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 437/872] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:13:48.993820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337f7ea-104e-473b-a502-1733c508ff55 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:48.994113 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337f7ea-104e-473b-a502-1733c508ff55 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:48.994396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337f7ea-104e-473b-a502-1733c508ff55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:49.011666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f337f7ea-104e-473b-a502-1733c508ff55 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:49.013640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f337f7ea-104e-473b-a502-1733c508ff55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:49.040622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65249e22-2374-4eb1-8e37-b6c545199590 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:49.083377 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c72c66-99a9-4100-b340-4f9cd0262eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:49.084193 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c72c66-99a9-4100-b340-4f9cd0262eca None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:49.084731 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c72c66-99a9-4100-b340-4f9cd0262eca None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:49.085078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c72c66-99a9-4100-b340-4f9cd0262eca None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:49.085885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c72c66-99a9-4100-b340-4f9cd0262eca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:49.086196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c72c66-99a9-4100-b340-4f9cd0262eca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:49.087666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c72c66-99a9-4100-b340-4f9cd0262eca None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:49.092301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 438/873] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:48 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:49.100943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 436/874] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:49.107481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337f7ea-104e-473b-a502-1733c508ff55 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:49.113553 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c72c66-99a9-4100-b340-4f9cd0262eca None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:49.234325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c72c66-99a9-4100-b340-4f9cd0262eca None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:49.247535 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337f7ea-104e-473b-a502-1733c508ff55 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:49.274822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d253e92-39bf-4acb-8184-8c38443da4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:49.275497 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d253e92-39bf-4acb-8184-8c38443da4ec None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:49.275782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d253e92-39bf-4acb-8184-8c38443da4ec None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:49.276078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d253e92-39bf-4acb-8184-8c38443da4ec None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:49.277713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d253e92-39bf-4acb-8184-8c38443da4ec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:49.278176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d253e92-39bf-4acb-8184-8c38443da4ec None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:49.289961 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559799af-6810-4434-9ca8-2e2e34397dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:49.290640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559799af-6810-4434-9ca8-2e2e34397dbf None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:49.290923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559799af-6810-4434-9ca8-2e2e34397dbf None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:49.291339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559799af-6810-4434-9ca8-2e2e34397dbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:49.292798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559799af-6810-4434-9ca8-2e2e34397dbf None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=c46e7cc6407f44f08a659288e3ad0767) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:49.292798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559799af-6810-4434-9ca8-2e2e34397dbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:49.293939 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559799af-6810-4434-9ca8-2e2e34397dbf None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:49.295837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 437/875] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:13:49 2022] POST /identity/v3/services => generated 251 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:13:49.309093 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 439/876] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:49 2022] GET /identity/v3/role_assignments?user.id=c46e7cc6407f44f08a659288e3ad0767&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=e59113c0366049e889df3f55eed90c72 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:50.321119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123751e6-f09a-4741-b8ec-d5c449c18b40 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:50.321644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123751e6-f09a-4741-b8ec-d5c449c18b40 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:50.321750 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123751e6-f09a-4741-b8ec-d5c449c18b40 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:50.323265 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 438/877] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:50.329741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d948bd5-e433-4dc3-b0bd-c43b285655da None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:50.330140 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d948bd5-e433-4dc3-b0bd-c43b285655da None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:50.330305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d948bd5-e433-4dc3-b0bd-c43b285655da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:50.346871 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-0d948bd5-e433-4dc3-b0bd-c43b285655da None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:50.348214 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0d948bd5-e433-4dc3-b0bd-c43b285655da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:50.425810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 440/878] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:50.448450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123751e6-f09a-4741-b8ec-d5c449c18b40 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:50.558351 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123751e6-f09a-4741-b8ec-d5c449c18b40 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:50.593071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37839c-8183-44d7-a03a-847c8bd06d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:50.593813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37839c-8183-44d7-a03a-847c8bd06d72 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:50.594093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37839c-8183-44d7-a03a-847c8bd06d72 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:50.594364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37839c-8183-44d7-a03a-847c8bd06d72 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:50.595016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37839c-8183-44d7-a03a-847c8bd06d72 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:50.596420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37839c-8183-44d7-a03a-847c8bd06d72 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-1f37839c-8183-44d7-a03a-847c8bd06d72 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:50.609373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:50.610402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:50.611997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 439/879] 10.222.0.44 () {56 vars in 1132 bytes} [Mon May 9 04:13:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:50.616667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d948bd5-e433-4dc3-b0bd-c43b285655da None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:50.713905 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d948bd5-e433-4dc3-b0bd-c43b285655da None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:50.760211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32ca9f-81c6-480a-a145-81e9111db6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:50.760891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32ca9f-81c6-480a-a145-81e9111db6a3 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:50.761176 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32ca9f-81c6-480a-a145-81e9111db6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:50.761456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32ca9f-81c6-480a-a145-81e9111db6a3 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:50.762178 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32ca9f-81c6-480a-a145-81e9111db6a3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:50.762646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32ca9f-81c6-480a-a145-81e9111db6a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:50.764226 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32ca9f-81c6-480a-a145-81e9111db6a3 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:50.769433 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 441/880] 10.222.0.44 () {56 vars in 1126 bytes} [Mon May 9 04:13:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:50.774966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37839c-8183-44d7-a03a-847c8bd06d72 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:50.873040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37839c-8183-44d7-a03a-847c8bd06d72 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:50.906811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0515134-c063-44c1-bea2-e0e93e9f46b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:50.907482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0515134-c063-44c1-bea2-e0e93e9f46b6 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:50.907761 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0515134-c063-44c1-bea2-e0e93e9f46b6 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:50.908067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0515134-c063-44c1-bea2-e0e93e9f46b6 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:50.908776 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0515134-c063-44c1-bea2-e0e93e9f46b6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:50.909079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0515134-c063-44c1-bea2-e0e93e9f46b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:50.910445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0515134-c063-44c1-bea2-e0e93e9f46b6 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:50.915532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 440/881] 10.222.0.44 () {56 vars in 1126 bytes} [Mon May 9 04:13:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:50.921834 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32ca9f-81c6-480a-a145-81e9111db6a3 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:51.022199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32ca9f-81c6-480a-a145-81e9111db6a3 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:51.055008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17565885-51d2-4057-b784-c8b2d45f1dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:51.055677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565885-51d2-4057-b784-c8b2d45f1dfd None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:51.055974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565885-51d2-4057-b784-c8b2d45f1dfd None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:51.056254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565885-51d2-4057-b784-c8b2d45f1dfd None admin] PATH_INFO: `/v3/endpoints` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:51.057044 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17565885-51d2-4057-b784-c8b2d45f1dfd None admin] RBAC: Adding query filter params (interface=public, service_id=459c195a1de04bae8b23b9a470bd030b, region_id=RegionOne) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:51.057353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17565885-51d2-4057-b784-c8b2d45f1dfd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:51.058709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17565885-51d2-4057-b784-c8b2d45f1dfd None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:51.064612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 442/882] 10.222.0.44 () {56 vars in 1251 bytes} [Mon May 9 04:13:50 2022] GET /identity/v3/endpoints?service_id=459c195a1de04bae8b23b9a470bd030b&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:51.069971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0515134-c063-44c1-bea2-e0e93e9f46b6 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:51.174620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0515134-c063-44c1-bea2-e0e93e9f46b6 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:51.208778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e7fca-8cc1-4e2e-92e5-648c9e605a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:51.209507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e7fca-8cc1-4e2e-92e5-648c9e605a82 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:51.209789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e7fca-8cc1-4e2e-92e5-648c9e605a82 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:51.210064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e7fca-8cc1-4e2e-92e5-648c9e605a82 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:51.210789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e7fca-8cc1-4e2e-92e5-648c9e605a82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:51.212190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e7fca-8cc1-4e2e-92e5-648c9e605a82 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:51.218561 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 441/883] 10.222.0.44 () {56 vars in 1087 bytes} [Mon May 9 04:13:51 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:13:51.463397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555bd02-4010-4224-a8cb-e66ab2aa37e8 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:51.463929 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555bd02-4010-4224-a8cb-e66ab2aa37e8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:51.464170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555bd02-4010-4224-a8cb-e66ab2aa37e8 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:51.465959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 443/884] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:51.471621 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d195ab-c70d-4f86-b9a5-9a9e351b238a None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:51.472083 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d195ab-c70d-4f86-b9a5-9a9e351b238a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:51.472429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d195ab-c70d-4f86-b9a5-9a9e351b238a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:51.489787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-32d195ab-c70d-4f86-b9a5-9a9e351b238a None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:51.490958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-32d195ab-c70d-4f86-b9a5-9a9e351b238a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:51.563944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 442/885] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:51.587370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555bd02-4010-4224-a8cb-e66ab2aa37e8 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:51.600986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412c99b-e9b2-4c1f-8288-ec00a2bd9146 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:51.601304 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412c99b-e9b2-4c1f-8288-ec00a2bd9146 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:51.602219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412c99b-e9b2-4c1f-8288-ec00a2bd9146 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:51.604140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 443/886] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:51.608712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11c0c2-32a1-4180-8195-933c65eec341 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:51.608998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11c0c2-32a1-4180-8195-933c65eec341 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:51.609275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11c0c2-32a1-4180-8195-933c65eec341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:51.628883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-fe11c0c2-32a1-4180-8195-933c65eec341 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:51.633221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-fe11c0c2-32a1-4180-8195-933c65eec341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:51.704391 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555bd02-4010-4224-a8cb-e66ab2aa37e8 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:51.716399 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 444/887] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:13:51.730243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f9039-a5f7-4201-9e86-7cfb1bc31722 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:51.730535 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f9039-a5f7-4201-9e86-7cfb1bc31722 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:51.730816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f9039-a5f7-4201-9e86-7cfb1bc31722 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:51.737063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 445/888] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:13:51.738318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaac1c0-81e8-4cb5-a50d-b3f9147eda92 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:51.738608 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaac1c0-81e8-4cb5-a50d-b3f9147eda92 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:51.738890 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaac1c0-81e8-4cb5-a50d-b3f9147eda92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:51.749877 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657ddff-7a57-42da-b0b2-1380b3efc5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:51.751068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657ddff-7a57-42da-b0b2-1380b3efc5fa None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:51.751428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657ddff-7a57-42da-b0b2-1380b3efc5fa None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:51.751707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657ddff-7a57-42da-b0b2-1380b3efc5fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:51.757725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-feaac1c0-81e8-4cb5-a50d-b3f9147eda92 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:51.758967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-feaac1c0-81e8-4cb5-a50d-b3f9147eda92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:51.763185 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657ddff-7a57-42da-b0b2-1380b3efc5fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-9657ddff-7a57-42da-b0b2-1380b3efc5fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:51.771346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:51.772388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:51.773221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:51.773221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:51.773221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:51.773221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:51.773221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:51.773221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:51.773221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:51.773221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:51.773221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:51.773221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:51.773221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 04:13:51.773221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:51.773221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 444/889] 10.222.0.44 () {56 vars in 1120 bytes} [Mon May 9 04:13:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:51.774536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657ddff-7a57-42da-b0b2-1380b3efc5fa None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:51.833636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 446/890] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:51.834482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaac1c0-81e8-4cb5-a50d-b3f9147eda92 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:51.887530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657ddff-7a57-42da-b0b2-1380b3efc5fa None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:51.925109 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6f731-f674-44ff-8612-2310ca82ee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:51.926491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6f731-f674-44ff-8612-2310ca82ee3c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:51.926776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6f731-f674-44ff-8612-2310ca82ee3c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:51.927051 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6f731-f674-44ff-8612-2310ca82ee3c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:51.928168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6f731-f674-44ff-8612-2310ca82ee3c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:51.929828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6f731-f674-44ff-8612-2310ca82ee3c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:51.930663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaac1c0-81e8-4cb5-a50d-b3f9147eda92 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-70a6f731-f674-44ff-8612-2310ca82ee3c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:51.942238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:51.943297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:51.944779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 445/891] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:13:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:51.945603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6f731-f674-44ff-8612-2310ca82ee3c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:51.970689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5b02c-76da-48e1-a85f-5f6f63c3a7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:51.971289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5b02c-76da-48e1-a85f-5f6f63c3a7ce None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:51.971600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5b02c-76da-48e1-a85f-5f6f63c3a7ce None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:51.971882 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5b02c-76da-48e1-a85f-5f6f63c3a7ce None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:51.972661 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5b02c-76da-48e1-a85f-5f6f63c3a7ce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:51.972967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5b02c-76da-48e1-a85f-5f6f63c3a7ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:51.974336 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5b02c-76da-48e1-a85f-5f6f63c3a7ce None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:51.982893 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 447/892] 10.222.0.44 () {56 vars in 1115 bytes} [Mon May 9 04:13:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:51.983718 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5b02c-76da-48e1-a85f-5f6f63c3a7ce None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:52.045705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6f731-f674-44ff-8612-2310ca82ee3c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:52.079512 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975add6d-9e1b-4fca-9443-ea6d447c3e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:52.080712 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975add6d-9e1b-4fca-9443-ea6d447c3e0a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:52.081194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975add6d-9e1b-4fca-9443-ea6d447c3e0a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:52.081665 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975add6d-9e1b-4fca-9443-ea6d447c3e0a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:52.082493 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975add6d-9e1b-4fca-9443-ea6d447c3e0a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:52.083112 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5b02c-76da-48e1-a85f-5f6f63c3a7ce None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:52.084256 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975add6d-9e1b-4fca-9443-ea6d447c3e0a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-975add6d-9e1b-4fca-9443-ea6d447c3e0a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:52.096205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:52.097194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:13:52.098200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:52.100644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 446/893] 10.222.0.44 () {56 vars in 1126 bytes} [Mon May 9 04:13:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:52.101539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975add6d-9e1b-4fca-9443-ea6d447c3e0a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:52.119537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124f093-5b76-4435-9870-96339d55e7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:52.120233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124f093-5b76-4435-9870-96339d55e7c3 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:52.120519 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124f093-5b76-4435-9870-96339d55e7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:52.120799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124f093-5b76-4435-9870-96339d55e7c3 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:52.121505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124f093-5b76-4435-9870-96339d55e7c3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:52.121810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124f093-5b76-4435-9870-96339d55e7c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:52.123931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124f093-5b76-4435-9870-96339d55e7c3 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:52.132828 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 448/894] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:13:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:52.132828 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124f093-5b76-4435-9870-96339d55e7c3 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:52.214671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975add6d-9e1b-4fca-9443-ea6d447c3e0a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:52.223921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124f093-5b76-4435-9870-96339d55e7c3 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:52.248988 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a6059-de07-4343-b1a6-d4ef297851ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:52.250310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a6059-de07-4343-b1a6-d4ef297851ff None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:52.250633 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a6059-de07-4343-b1a6-d4ef297851ff None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:52.250922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a6059-de07-4343-b1a6-d4ef297851ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:52.257909 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b310ccf-81db-4429-8447-6a5c41417c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:52.258365 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b310ccf-81db-4429-8447-6a5c41417c96 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:52.258645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b310ccf-81db-4429-8447-6a5c41417c96 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:52.258923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b310ccf-81db-4429-8447-6a5c41417c96 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:52.259616 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b310ccf-81db-4429-8447-6a5c41417c96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:52.259937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b310ccf-81db-4429-8447-6a5c41417c96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:52.261126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a6059-de07-4343-b1a6-d4ef297851ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:52.261294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b310ccf-81db-4429-8447-6a5c41417c96 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:52.262547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a6059-de07-4343-b1a6-d4ef297851ff None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:52.270456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 449/895] 10.222.0.44 () {56 vars in 1122 bytes} [Mon May 9 04:13:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:52.270456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b310ccf-81db-4429-8447-6a5c41417c96 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-a61a6059-de07-4343-b1a6-d4ef297851ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:52.280190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:52.281106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:52.282007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:52.282007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:52.282007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:52.282007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:52.282007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 447/896] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:52.282299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a6059-de07-4343-b1a6-d4ef297851ff None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:52.373525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b310ccf-81db-4429-8447-6a5c41417c96 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:52.384996 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a6059-de07-4343-b1a6-d4ef297851ff None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:52.407041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f22a0b-26ff-493a-9488-87a202119b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:52.407709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f22a0b-26ff-493a-9488-87a202119b2b None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:52.408078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f22a0b-26ff-493a-9488-87a202119b2b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:52.408361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f22a0b-26ff-493a-9488-87a202119b2b None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:52.409060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f22a0b-26ff-493a-9488-87a202119b2b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:52.409362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f22a0b-26ff-493a-9488-87a202119b2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:52.410724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f22a0b-26ff-493a-9488-87a202119b2b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:52.415558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 450/897] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:13:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:52.416282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f22a0b-26ff-493a-9488-87a202119b2b None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:52.422555 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627a8b5-8559-4db5-9791-11fb2e9d5e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:52.423241 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627a8b5-8559-4db5-9791-11fb2e9d5e20 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:52.423530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627a8b5-8559-4db5-9791-11fb2e9d5e20 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:52.423808 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627a8b5-8559-4db5-9791-11fb2e9d5e20 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:52.424552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627a8b5-8559-4db5-9791-11fb2e9d5e20 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:52.425098 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627a8b5-8559-4db5-9791-11fb2e9d5e20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:52.426205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627a8b5-8559-4db5-9791-11fb2e9d5e20 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:52.440219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 448/898] 10.222.0.44 () {56 vars in 1122 bytes} [Mon May 9 04:13:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:52.440219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627a8b5-8559-4db5-9791-11fb2e9d5e20 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:52.539167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f22a0b-26ff-493a-9488-87a202119b2b None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:52.555269 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627a8b5-8559-4db5-9791-11fb2e9d5e20 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:52.577082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ffca9-dcae-4891-8f10-e77e18e76ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:52.577894 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ffca9-dcae-4891-8f10-e77e18e76ef4 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:52.578285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ffca9-dcae-4891-8f10-e77e18e76ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:52.578663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ffca9-dcae-4891-8f10-e77e18e76ef4 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:52.579782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ffca9-dcae-4891-8f10-e77e18e76ef4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:52.580223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ffca9-dcae-4891-8f10-e77e18e76ef4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:52.581737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ffca9-dcae-4891-8f10-e77e18e76ef4 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:52.592644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be620a-0713-4b2f-a60c-047b6f348920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:52.593320 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be620a-0713-4b2f-a60c-047b6f348920 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:52.593606 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be620a-0713-4b2f-a60c-047b6f348920 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:52.593885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be620a-0713-4b2f-a60c-047b6f348920 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:52.594685 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be620a-0713-4b2f-a60c-047b6f348920 None admin] RBAC: Adding query filter params (interface=public, service_id=98ff4f04ab05479f93845f545c8ca2c0, region_id=RegionOne) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:52.594996 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be620a-0713-4b2f-a60c-047b6f348920 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:52.596489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be620a-0713-4b2f-a60c-047b6f348920 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:52.599794 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 451/899] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:52.600781 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ffca9-dcae-4891-8f10-e77e18e76ef4 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:52.602687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 449/900] 10.222.0.44 () {56 vars in 1251 bytes} [Mon May 9 04:13:52 2022] GET /identity/v3/endpoints?service_id=98ff4f04ab05479f93845f545c8ca2c0&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:52.609012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be620a-0713-4b2f-a60c-047b6f348920 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:52.730384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be620a-0713-4b2f-a60c-047b6f348920 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:52.732084 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ffca9-dcae-4891-8f10-e77e18e76ef4 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:52.777756 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889c78c-820e-4e50-8342-c543b95cfbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:52.779631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c78c-820e-4e50-8342-c543b95cfbb8 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:52.780174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c78c-820e-4e50-8342-c543b95cfbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:52.780351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8247c-308d-4969-ba66-a279df99cd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:52.780832 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c78c-820e-4e50-8342-c543b95cfbb8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:52.781019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8247c-308d-4969-ba66-a279df99cd8e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:52.781259 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8247c-308d-4969-ba66-a279df99cd8e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:52.781535 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8247c-308d-4969-ba66-a279df99cd8e None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:52.782215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8247c-308d-4969-ba66-a279df99cd8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:52.783680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8247c-308d-4969-ba66-a279df99cd8e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:52.789490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889c78c-820e-4e50-8342-c543b95cfbb8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:52.789759 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 450/901] 10.222.0.44 () {56 vars in 1087 bytes} [Mon May 9 04:13:52 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:13:52.791444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889c78c-820e-4e50-8342-c543b95cfbb8 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-0889c78c-820e-4e50-8342-c543b95cfbb8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:52.812120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:52.813172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:52.814131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:52.814131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:52.814131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:52.814131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:52.814131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:52.814131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:52.814131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:52.814131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:52.814131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 04:13:52.814131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:52.814131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 452/902] 10.222.0.44 () {56 vars in 1117 bytes} [Mon May 9 04:13:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:52.819221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8247c-308d-4969-ba66-a279df99cd8e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:52.928397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8247c-308d-4969-ba66-a279df99cd8e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:52.966860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f286f31-32de-45b3-aae6-217af15b1ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:52.967958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f286f31-32de-45b3-aae6-217af15b1ec5 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:52.968488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f286f31-32de-45b3-aae6-217af15b1ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:52.968988 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f286f31-32de-45b3-aae6-217af15b1ec5 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:52.970141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f286f31-32de-45b3-aae6-217af15b1ec5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:52.970679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f286f31-32de-45b3-aae6-217af15b1ec5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:52.972366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f286f31-32de-45b3-aae6-217af15b1ec5 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:52.992295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 451/903] 10.222.0.44 () {56 vars in 1149 bytes} [Mon May 9 04:13:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:52.998252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889c78c-820e-4e50-8342-c543b95cfbb8 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:53.105322 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889c78c-820e-4e50-8342-c543b95cfbb8 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:53.141170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab09844-84c2-4c29-bb02-df73b68349f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:53.142197 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab09844-84c2-4c29-bb02-df73b68349f3 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:53.142731 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab09844-84c2-4c29-bb02-df73b68349f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:53.143226 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab09844-84c2-4c29-bb02-df73b68349f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:53.210921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab09844-84c2-4c29-bb02-df73b68349f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:53.212773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab09844-84c2-4c29-bb02-df73b68349f3 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:53.317699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 453/904] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:13:52 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 04:13:53.325048 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f286f31-32de-45b3-aae6-217af15b1ec5 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:53.427652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f286f31-32de-45b3-aae6-217af15b1ec5 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:53.461738 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4bbb9-00e8-44a6-876b-45d51bdf425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:53.462537 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4bbb9-00e8-44a6-876b-45d51bdf425f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:53.462819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4bbb9-00e8-44a6-876b-45d51bdf425f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:53.463220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4bbb9-00e8-44a6-876b-45d51bdf425f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:53.473984 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4bbb9-00e8-44a6-876b-45d51bdf425f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:53.475426 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4bbb9-00e8-44a6-876b-45d51bdf425f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:53.476446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8eb07-8f5b-465f-a466-c5fb0611859a None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:53.476794 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8eb07-8f5b-465f-a466-c5fb0611859a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:53.477077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8eb07-8f5b-465f-a466-c5fb0611859a None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:53.478650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 454/905] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:53.484299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e315a-196e-4414-b51f-42c8752e4d6a None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:53.484587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e315a-196e-4414-b51f-42c8752e4d6a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:53.484861 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e315a-196e-4414-b51f-42c8752e4d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-abd4bbb9-00e8-44a6-876b-45d51bdf425f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:53.493912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:53.494820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:53.495744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:53.495744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:53.495744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:53.495744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:53.495744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 452/906] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:53.500929 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4bbb9-00e8-44a6-876b-45d51bdf425f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:53.506045 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-178e315a-196e-4414-b51f-42c8752e4d6a None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:53.507235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-178e315a-196e-4414-b51f-42c8752e4d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:53.588690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 455/907] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:13:53.611193 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e315a-196e-4414-b51f-42c8752e4d6a None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:53.618424 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4bbb9-00e8-44a6-876b-45d51bdf425f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:53.654159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec97a01-e080-4eb1-a447-43cd128c245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:53.654159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec97a01-e080-4eb1-a447-43cd128c245c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:53.654501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec97a01-e080-4eb1-a447-43cd128c245c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:53.654843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec97a01-e080-4eb1-a447-43cd128c245c None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:53.655662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec97a01-e080-4eb1-a447-43cd128c245c None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:53.656092 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec97a01-e080-4eb1-a447-43cd128c245c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:53.657542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec97a01-e080-4eb1-a447-43cd128c245c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:53.669986 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 453/908] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:53.677081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec97a01-e080-4eb1-a447-43cd128c245c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:53.729473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e315a-196e-4414-b51f-42c8752e4d6a None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:53.769156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abfee0-d91d-4b58-ada8-7dfe4cf4e41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:53.770297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abfee0-d91d-4b58-ada8-7dfe4cf4e41b None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:53.770297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abfee0-d91d-4b58-ada8-7dfe4cf4e41b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:53.770569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abfee0-d91d-4b58-ada8-7dfe4cf4e41b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:53.771188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abfee0-d91d-4b58-ada8-7dfe4cf4e41b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:53.772905 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abfee0-d91d-4b58-ada8-7dfe4cf4e41b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-f1abfee0-d91d-4b58-ada8-7dfe4cf4e41b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:53.785477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:53.786446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 04:13:53.787297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:53.791711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 456/909] 10.222.0.44 () {56 vars in 1132 bytes} [Mon May 9 04:13:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:53.792760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abfee0-d91d-4b58-ada8-7dfe4cf4e41b None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:53.802495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec97a01-e080-4eb1-a447-43cd128c245c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:53.846009 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27d8fc-188a-4d78-a6a0-a1addaa4d021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:53.847739 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27d8fc-188a-4d78-a6a0-a1addaa4d021 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:53.848273 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27d8fc-188a-4d78-a6a0-a1addaa4d021 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:53.848916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27d8fc-188a-4d78-a6a0-a1addaa4d021 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:53.858521 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27d8fc-188a-4d78-a6a0-a1addaa4d021 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:53.861134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27d8fc-188a-4d78-a6a0-a1addaa4d021 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-4a27d8fc-188a-4d78-a6a0-a1addaa4d021 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:53.882441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:53.883413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:53.884425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:53.884425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:53.884425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:53.884425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:53.884425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:53.884425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:53.884425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:53.884425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:53.891802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 454/910] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:53.892813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abfee0-d91d-4b58-ada8-7dfe4cf4e41b None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:53.893161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27d8fc-188a-4d78-a6a0-a1addaa4d021 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:53.925869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f2de2-20a2-46d2-9567-5b3fd84d6678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:53.926589 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2de2-20a2-46d2-9567-5b3fd84d6678 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:53.926867 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2de2-20a2-46d2-9567-5b3fd84d6678 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:53.927185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2de2-20a2-46d2-9567-5b3fd84d6678 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:53.927906 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f2de2-20a2-46d2-9567-5b3fd84d6678 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:53.928222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f2de2-20a2-46d2-9567-5b3fd84d6678 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:53.929579 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f2de2-20a2-46d2-9567-5b3fd84d6678 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:53.939641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 457/911] 10.222.0.44 () {56 vars in 1126 bytes} [Mon May 9 04:13:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:53.940605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f2de2-20a2-46d2-9567-5b3fd84d6678 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:54.002243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27d8fc-188a-4d78-a6a0-a1addaa4d021 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:54.038223 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531ec1d-6ed0-445e-8d0e-75dd8ba87b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:54.039258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531ec1d-6ed0-445e-8d0e-75dd8ba87b82 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:54.039748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531ec1d-6ed0-445e-8d0e-75dd8ba87b82 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:54.039976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f2de2-20a2-46d2-9567-5b3fd84d6678 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:54.040455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531ec1d-6ed0-445e-8d0e-75dd8ba87b82 None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:54.041455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531ec1d-6ed0-445e-8d0e-75dd8ba87b82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:54.041968 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531ec1d-6ed0-445e-8d0e-75dd8ba87b82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:54.043609 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531ec1d-6ed0-445e-8d0e-75dd8ba87b82 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:54.062700 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 455/912] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:54.063952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531ec1d-6ed0-445e-8d0e-75dd8ba87b82 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:54.074304 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fda27-b61c-4cdb-b0d2-219c6be183bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:54.075000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fda27-b61c-4cdb-b0d2-219c6be183bf None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:54.075323 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fda27-b61c-4cdb-b0d2-219c6be183bf None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:54.075768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fda27-b61c-4cdb-b0d2-219c6be183bf None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:54.076391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fda27-b61c-4cdb-b0d2-219c6be183bf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:54.076799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fda27-b61c-4cdb-b0d2-219c6be183bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:54.078171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fda27-b61c-4cdb-b0d2-219c6be183bf None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:54.083770 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 458/913] 10.222.0.44 () {56 vars in 1126 bytes} [Mon May 9 04:13:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:54.089352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fda27-b61c-4cdb-b0d2-219c6be183bf None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:54.171915 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531ec1d-6ed0-445e-8d0e-75dd8ba87b82 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:54.188942 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fda27-b61c-4cdb-b0d2-219c6be183bf None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:54.208432 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe0383-78c0-4d7e-ab69-2407a5458c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:54.209408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe0383-78c0-4d7e-ab69-2407a5458c75 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:54.209927 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe0383-78c0-4d7e-ab69-2407a5458c75 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:54.210420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe0383-78c0-4d7e-ab69-2407a5458c75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:54.211491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe0383-78c0-4d7e-ab69-2407a5458c75 None admin] RBAC: Adding query filter params (role.id=7de5200fcd0943d9834b6e12861d5941, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=ee0055c75b91471a99d9bb18248839ea) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:54.212040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe0383-78c0-4d7e-ab69-2407a5458c75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:54.213679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe0383-78c0-4d7e-ab69-2407a5458c75 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:54.228147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e63d6-2b77-4a00-867c-6e75625f9499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:54.228147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e63d6-2b77-4a00-867c-6e75625f9499 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:54.228147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e63d6-2b77-4a00-867c-6e75625f9499 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:54.228147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e63d6-2b77-4a00-867c-6e75625f9499 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:54.228550 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e63d6-2b77-4a00-867c-6e75625f9499 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:54.228550 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e63d6-2b77-4a00-867c-6e75625f9499 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:54.228945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 456/914] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:13:54 2022] GET /identity/v3/role_assignments?user.id=ee0055c75b91471a99d9bb18248839ea&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=7de5200fcd0943d9834b6e12861d5941 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:54.251727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 459/915] 10.222.0.44 () {60 vars in 1144 bytes} [Mon May 9 04:13:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:13:54.749935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bacde-ba7e-49db-86b8-7e4acf5eb9cd None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:54.750447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bacde-ba7e-49db-86b8-7e4acf5eb9cd None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:54.750554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bacde-ba7e-49db-86b8-7e4acf5eb9cd None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:54.752136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 457/916] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:54.760607 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eda61-4f6a-4880-9ed0-02df8dd73dd2 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:54.761055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eda61-4f6a-4880-9ed0-02df8dd73dd2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:54.761299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eda61-4f6a-4880-9ed0-02df8dd73dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:54.777987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-492eda61-4f6a-4880-9ed0-02df8dd73dd2 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:54.779934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-492eda61-4f6a-4880-9ed0-02df8dd73dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:54.854616 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 460/917] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:54.877639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bacde-ba7e-49db-86b8-7e4acf5eb9cd None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:54.994250 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bacde-ba7e-49db-86b8-7e4acf5eb9cd None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:55.030330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1abe3-e096-42bc-80de-987d7a95693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:55.030912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1abe3-e096-42bc-80de-987d7a95693f None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:55.031195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1abe3-e096-42bc-80de-987d7a95693f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:55.031467 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1abe3-e096-42bc-80de-987d7a95693f None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:55.032220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1abe3-e096-42bc-80de-987d7a95693f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:55.033575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1abe3-e096-42bc-80de-987d7a95693f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:55.049560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 458/918] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:13:54 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:13:55.111022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1f6e4-d5f1-4296-991c-f6f1bddef5dd None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:55.111356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1f6e4-d5f1-4296-991c-f6f1bddef5dd None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:55.111583 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1f6e4-d5f1-4296-991c-f6f1bddef5dd None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:55.113422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 461/919] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:55.119541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65265cf6-75a8-4a1d-b535-cc42025948d9 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:55.119868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65265cf6-75a8-4a1d-b535-cc42025948d9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:55.120131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65265cf6-75a8-4a1d-b535-cc42025948d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:55.138936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-65265cf6-75a8-4a1d-b535-cc42025948d9 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:55.140181 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-65265cf6-75a8-4a1d-b535-cc42025948d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:55.221120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 459/920] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:13:55.243789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1f6e4-d5f1-4296-991c-f6f1bddef5dd None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:55.342516 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1f6e4-d5f1-4296-991c-f6f1bddef5dd None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:55.375622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083733e6-75a8-41fb-9371-2ad278cf2dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:55.376487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083733e6-75a8-41fb-9371-2ad278cf2dba None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:55.376767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083733e6-75a8-41fb-9371-2ad278cf2dba None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:55.377042 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083733e6-75a8-41fb-9371-2ad278cf2dba None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:55.377740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083733e6-75a8-41fb-9371-2ad278cf2dba None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:55.379337 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083733e6-75a8-41fb-9371-2ad278cf2dba None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-083733e6-75a8-41fb-9371-2ad278cf2dba None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:55.391824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:55.392855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:55.393691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 462/921] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:13:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:55.399231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65265cf6-75a8-4a1d-b535-cc42025948d9 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:55.505218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65265cf6-75a8-4a1d-b535-cc42025948d9 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:55.539955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ceef0-0610-47e2-bd1b-be48d6ed1376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:55.540626 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ceef0-0610-47e2-bd1b-be48d6ed1376 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:55.540904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ceef0-0610-47e2-bd1b-be48d6ed1376 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:55.541174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ceef0-0610-47e2-bd1b-be48d6ed1376 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:55.541872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ceef0-0610-47e2-bd1b-be48d6ed1376 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:55.542233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ceef0-0610-47e2-bd1b-be48d6ed1376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:55.543586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ceef0-0610-47e2-bd1b-be48d6ed1376 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:55.549375 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 460/922] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:13:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:55.555148 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083733e6-75a8-41fb-9371-2ad278cf2dba None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:55.664543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083733e6-75a8-41fb-9371-2ad278cf2dba None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:55.698320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af3cf3-a668-4ae6-a4b7-c3c9d3871954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:55.699014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af3cf3-a668-4ae6-a4b7-c3c9d3871954 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:55.699306 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af3cf3-a668-4ae6-a4b7-c3c9d3871954 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:55.699579 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af3cf3-a668-4ae6-a4b7-c3c9d3871954 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:55.700338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af3cf3-a668-4ae6-a4b7-c3c9d3871954 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:55.700640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af3cf3-a668-4ae6-a4b7-c3c9d3871954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:55.701999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af3cf3-a668-4ae6-a4b7-c3c9d3871954 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:55.707319 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 463/923] 10.222.0.44 () {56 vars in 1114 bytes} [Mon May 9 04:13:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:55.712866 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ceef0-0610-47e2-bd1b-be48d6ed1376 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:55.811537 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ceef0-0610-47e2-bd1b-be48d6ed1376 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:55.845881 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0dc3a5-1bd0-4ecb-86da-60126b5da9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:55.846539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dc3a5-1bd0-4ecb-86da-60126b5da9cd None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:55.846815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dc3a5-1bd0-4ecb-86da-60126b5da9cd None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:55.847086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dc3a5-1bd0-4ecb-86da-60126b5da9cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:55.847824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0dc3a5-1bd0-4ecb-86da-60126b5da9cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:55.849196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0dc3a5-1bd0-4ecb-86da-60126b5da9cd None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:55.869357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 461/924] 10.222.0.44 () {60 vars in 1144 bytes} [Mon May 9 04:13:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:13:56.565173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcd306-e956-4cc1-9cd4-443a75d6fde2 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:56.565726 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcd306-e956-4cc1-9cd4-443a75d6fde2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:56.565726 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcd306-e956-4cc1-9cd4-443a75d6fde2 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:56.567365 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 464/925] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:56.575206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3029e75-9ade-4bbf-aa91-c0bfd99743a8 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:56.576030 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3029e75-9ade-4bbf-aa91-c0bfd99743a8 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:56.576309 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3029e75-9ade-4bbf-aa91-c0bfd99743a8 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:56.577844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 465/926] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:56.578134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e03abe-be3e-476f-98b6-0c95c85bec12 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:56.578642 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e03abe-be3e-476f-98b6-0c95c85bec12 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:56.579457 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e03abe-be3e-476f-98b6-0c95c85bec12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:56.582805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96d887-be45-49ce-a6ab-1db653b3aa40 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:56.583087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96d887-be45-49ce-a6ab-1db653b3aa40 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:56.583361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96d887-be45-49ce-a6ab-1db653b3aa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:56.598944 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-53e03abe-be3e-476f-98b6-0c95c85bec12 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:56.600488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-53e03abe-be3e-476f-98b6-0c95c85bec12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:56.601770 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-cb96d887-be45-49ce-a6ab-1db653b3aa40 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:56.603280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cb96d887-be45-49ce-a6ab-1db653b3aa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:56.674112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 462/927] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:56.691970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 466/928] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:13:56.696414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e03abe-be3e-476f-98b6-0c95c85bec12 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:56.709784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96d887-be45-49ce-a6ab-1db653b3aa40 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:56.795940 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e03abe-be3e-476f-98b6-0c95c85bec12 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:56.819279 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96d887-be45-49ce-a6ab-1db653b3aa40 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:56.830217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c142d-280a-48c2-900f-da541f52fc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:56.831686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c142d-280a-48c2-900f-da541f52fc66 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:56.832205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c142d-280a-48c2-900f-da541f52fc66 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:56.832678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c142d-280a-48c2-900f-da541f52fc66 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:56.843019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c142d-280a-48c2-900f-da541f52fc66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-6b8c142d-280a-48c2-900f-da541f52fc66 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:13:56.851364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:56.852496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:56.853331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:56.853331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:56.853331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:56.853331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:56.853331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:13:56.853331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:13:56.853331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:13:56.853331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:13:56.853331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:13:56.853331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:13:56.853331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 04:13:56.853331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:56.853969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4d773-5f7f-4f53-9a0e-2d2b76718e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:56.853969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4d773-5f7f-4f53-9a0e-2d2b76718e57 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:56.854112 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4d773-5f7f-4f53-9a0e-2d2b76718e57 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:56.854324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4d773-5f7f-4f53-9a0e-2d2b76718e57 None admin] PATH_INFO: `/v3/services/compute` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:56.855117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4d773-5f7f-4f53-9a0e-2d2b76718e57 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:56.857237 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4d773-5f7f-4f53-9a0e-2d2b76718e57 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:56.860344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 463/929] 10.222.0.44 () {56 vars in 1120 bytes} [Mon May 9 04:13:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:56.862679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c142d-280a-48c2-900f-da541f52fc66 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-1ee4d773-5f7f-4f53-9a0e-2d2b76718e57 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:56.870257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:56.871303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 04:13:56.872897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:56.879996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 467/930] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:56.884838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4d773-5f7f-4f53-9a0e-2d2b76718e57 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:56.964998 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c142d-280a-48c2-900f-da541f52fc66 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:57.003875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c788d9-17c1-4bc0-9691-a2cf26594082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:57.004657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c788d9-17c1-4bc0-9691-a2cf26594082 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:57.004883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c788d9-17c1-4bc0-9691-a2cf26594082 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:57.005173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c788d9-17c1-4bc0-9691-a2cf26594082 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:57.005907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c788d9-17c1-4bc0-9691-a2cf26594082 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:57.006209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c788d9-17c1-4bc0-9691-a2cf26594082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:57.007803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c788d9-17c1-4bc0-9691-a2cf26594082 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:57.017380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4d773-5f7f-4f53-9a0e-2d2b76718e57 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:57.020602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 464/931] 10.222.0.44 () {56 vars in 1115 bytes} [Mon May 9 04:13:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:57.021139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c788d9-17c1-4bc0-9691-a2cf26594082 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:57.057768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd4d03-5a88-4f19-b8e7-fca4517ac718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:57.058435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd4d03-5a88-4f19-b8e7-fca4517ac718 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:57.058716 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd4d03-5a88-4f19-b8e7-fca4517ac718 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:57.058990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd4d03-5a88-4f19-b8e7-fca4517ac718 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:57.059752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd4d03-5a88-4f19-b8e7-fca4517ac718 None admin] RBAC: Adding query filter params (name=compute) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:57.060084 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd4d03-5a88-4f19-b8e7-fca4517ac718 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:57.061459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd4d03-5a88-4f19-b8e7-fca4517ac718 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:57.070820 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 468/932] 10.222.0.44 () {56 vars in 1112 bytes} [Mon May 9 04:13:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:57.072866 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd4d03-5a88-4f19-b8e7-fca4517ac718 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:57.131847 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c788d9-17c1-4bc0-9691-a2cf26594082 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:57.166966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877de21-2beb-4693-8eae-3aefb9af28b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:57.168149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877de21-2beb-4693-8eae-3aefb9af28b1 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:57.168575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877de21-2beb-4693-8eae-3aefb9af28b1 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:57.169104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877de21-2beb-4693-8eae-3aefb9af28b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:57.180583 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e877de21-2beb-4693-8eae-3aefb9af28b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:57.182303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e877de21-2beb-4693-8eae-3aefb9af28b1 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:57.198228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd4d03-5a88-4f19-b8e7-fca4517ac718 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-e877de21-2beb-4693-8eae-3aefb9af28b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:57.200146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:57.201490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:57.202394 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:57.202394 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:57.202394 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:57.202394 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:57.204950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 465/933] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:57.213313 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877de21-2beb-4693-8eae-3aefb9af28b1 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:57.244079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfb0f5-6f85-4579-b61b-afd681494deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:57.244810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfb0f5-6f85-4579-b61b-afd681494deb None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:57.245095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfb0f5-6f85-4579-b61b-afd681494deb None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:57.245358 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfb0f5-6f85-4579-b61b-afd681494deb None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:57.246186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfb0f5-6f85-4579-b61b-afd681494deb None admin] RBAC: Adding query filter params (type=compute) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:57.246491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfb0f5-6f85-4579-b61b-afd681494deb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:57.248925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfb0f5-6f85-4579-b61b-afd681494deb None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:57.254236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 469/934] 10.222.0.44 () {56 vars in 1112 bytes} [Mon May 9 04:13:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:57.336926 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877de21-2beb-4693-8eae-3aefb9af28b1 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:57.373794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb7ec4-c60f-4986-8eee-053bcc5813c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:57.374477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb7ec4-c60f-4986-8eee-053bcc5813c8 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:57.374801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb7ec4-c60f-4986-8eee-053bcc5813c8 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:57.375082 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb7ec4-c60f-4986-8eee-053bcc5813c8 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:57.375864 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb7ec4-c60f-4986-8eee-053bcc5813c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:57.376203 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb7ec4-c60f-4986-8eee-053bcc5813c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:57.377570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb7ec4-c60f-4986-8eee-053bcc5813c8 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:57.391296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 466/935] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:57.397108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfb0f5-6f85-4579-b61b-afd681494deb None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:57.428880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3746c-721a-48e4-8ef9-85886c8085f8 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:57.429173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3746c-721a-48e4-8ef9-85886c8085f8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:57.429455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3746c-721a-48e4-8ef9-85886c8085f8 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:57.431069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 467/936] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:13:57.437184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cfe85-50a9-4b61-b8c1-6ea58157c1cb None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:57.437473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cfe85-50a9-4b61-b8c1-6ea58157c1cb None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:57.437753 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188cfe85-50a9-4b61-b8c1-6ea58157c1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:57.457856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-188cfe85-50a9-4b61-b8c1-6ea58157c1cb None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:57.459038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-188cfe85-50a9-4b61-b8c1-6ea58157c1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:57.503782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfb0f5-6f85-4579-b61b-afd681494deb None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:57.538261 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e79a8-772f-42da-beb7-2a7c21f4f294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:57.538979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 468/937] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:13:57.539091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e79a8-772f-42da-beb7-2a7c21f4f294 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:57.539343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e79a8-772f-42da-beb7-2a7c21f4f294 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:57.539614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e79a8-772f-42da-beb7-2a7c21f4f294 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:57.554153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e79a8-772f-42da-beb7-2a7c21f4f294 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:57.555653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e79a8-772f-42da-beb7-2a7c21f4f294 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:57.563770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cfe85-50a9-4b61-b8c1-6ea58157c1cb None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-444e79a8-772f-42da-beb7-2a7c21f4f294 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:13:57.570581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:57.571915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:57.573450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:57.573450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:57.573450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:13:57.573450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:13:57.573450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:13:57.573450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:13:57.573450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:13:57.573450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:13:57.573450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 04:13:57.573450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:57.573450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 470/938] 10.222.0.44 () {56 vars in 1117 bytes} [Mon May 9 04:13:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:57.579346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e79a8-772f-42da-beb7-2a7c21f4f294 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:57.680826 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e79a8-772f-42da-beb7-2a7c21f4f294 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:57.681130 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188cfe85-50a9-4b61-b8c1-6ea58157c1cb None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:57.715173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb3e10-f004-492e-a33f-7f8f323ff21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:57.715531 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c6452-af09-4431-b466-2914d6db24ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:57.716457 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb3e10-f004-492e-a33f-7f8f323ff21d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:57.716726 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb3e10-f004-492e-a33f-7f8f323ff21d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:57.716891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c6452-af09-4431-b466-2914d6db24ad None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:57.717042 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb3e10-f004-492e-a33f-7f8f323ff21d None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:57.717147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c6452-af09-4431-b466-2914d6db24ad None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:57.719769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c6452-af09-4431-b466-2914d6db24ad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:57.720018 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb3e10-f004-492e-a33f-7f8f323ff21d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:57.720190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c6452-af09-4431-b466-2914d6db24ad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:57.720337 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb3e10-f004-492e-a33f-7f8f323ff21d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:57.720504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c6452-af09-4431-b466-2914d6db24ad None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:57.720654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb3e10-f004-492e-a33f-7f8f323ff21d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-d99c6452-af09-4431-b466-2914d6db24ad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:57.731506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:57.733869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:13:57.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 469/939] 10.222.0.44 () {56 vars in 1126 bytes} [Mon May 9 04:13:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:57.738552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c6452-af09-4431-b466-2914d6db24ad None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:57.743807 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 471/940] 10.222.0.44 () {56 vars in 1149 bytes} [Mon May 9 04:13:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:57.746404 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb3e10-f004-492e-a33f-7f8f323ff21d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:57.855154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb3e10-f004-492e-a33f-7f8f323ff21d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:57.874464 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c6452-af09-4431-b466-2914d6db24ad None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:57.899612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb486cc9-e051-4dbc-8291-230b991018a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:57.900430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb486cc9-e051-4dbc-8291-230b991018a2 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:57.900706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb486cc9-e051-4dbc-8291-230b991018a2 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:57.900975 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb486cc9-e051-4dbc-8291-230b991018a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:57.913062 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb486cc9-e051-4dbc-8291-230b991018a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:57.914506 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb486cc9-e051-4dbc-8291-230b991018a2 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:57.916247 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5127815-67ef-41cc-8c80-ef32a165a60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:57.917505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5127815-67ef-41cc-8c80-ef32a165a60f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:57.918026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5127815-67ef-41cc-8c80-ef32a165a60f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:57.918508 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5127815-67ef-41cc-8c80-ef32a165a60f None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:57.919494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5127815-67ef-41cc-8c80-ef32a165a60f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:57.920051 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5127815-67ef-41cc-8c80-ef32a165a60f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:57.923296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5127815-67ef-41cc-8c80-ef32a165a60f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:57.931226 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 470/941] 10.222.0.44 () {56 vars in 1122 bytes} [Mon May 9 04:13:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-eb486cc9-e051-4dbc-8291-230b991018a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:13:57.936007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:57.936897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:57.937783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:13:57.937783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:13:57.937783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:13:57.937783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:57.937783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 472/942] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:57.946698 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5127815-67ef-41cc-8c80-ef32a165a60f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:57.947064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb486cc9-e051-4dbc-8291-230b991018a2 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:58.054759 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5127815-67ef-41cc-8c80-ef32a165a60f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:58.089501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bfbb1-7fc4-4a2f-927b-772b8ef9ce84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:58.090136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bfbb1-7fc4-4a2f-927b-772b8ef9ce84 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:58.090417 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bfbb1-7fc4-4a2f-927b-772b8ef9ce84 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:58.090769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bfbb1-7fc4-4a2f-927b-772b8ef9ce84 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:58.091586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153bfbb1-7fc4-4a2f-927b-772b8ef9ce84 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:58.091826 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153bfbb1-7fc4-4a2f-927b-772b8ef9ce84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:58.093157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153bfbb1-7fc4-4a2f-927b-772b8ef9ce84 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:58.097998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb486cc9-e051-4dbc-8291-230b991018a2 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:58.102702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 471/943] 10.222.0.44 () {56 vars in 1122 bytes} [Mon May 9 04:13:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:58.103989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bfbb1-7fc4-4a2f-927b-772b8ef9ce84 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:58.142864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b493e-36eb-4761-80ff-f15112f996cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:58.144138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b493e-36eb-4761-80ff-f15112f996cf None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:58.144669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b493e-36eb-4761-80ff-f15112f996cf None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:58.145148 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b493e-36eb-4761-80ff-f15112f996cf None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:58.146490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b493e-36eb-4761-80ff-f15112f996cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:58.147064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b493e-36eb-4761-80ff-f15112f996cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:58.149331 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b493e-36eb-4761-80ff-f15112f996cf None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:58.166658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 473/944] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:58.168030 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b493e-36eb-4761-80ff-f15112f996cf None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:58.223713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bfbb1-7fc4-4a2f-927b-772b8ef9ce84 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:58.257022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5919c-7b14-434f-9cee-3548a107ebb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:58.257770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5919c-7b14-434f-9cee-3548a107ebb6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:58.258055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5919c-7b14-434f-9cee-3548a107ebb6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:58.258331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5919c-7b14-434f-9cee-3548a107ebb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:58.259124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5919c-7b14-434f-9cee-3548a107ebb6 None admin] RBAC: Adding query filter params (interface=public, service_id=99d9c58a99484ca8bc37de74040c2ec4, region_id=RegionOne) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:58.259435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5919c-7b14-434f-9cee-3548a107ebb6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:58.260814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5919c-7b14-434f-9cee-3548a107ebb6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:58.265746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b493e-36eb-4761-80ff-f15112f996cf None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:58.272025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 472/945] 10.222.0.44 () {56 vars in 1251 bytes} [Mon May 9 04:13:58 2022] GET /identity/v3/endpoints?service_id=99d9c58a99484ca8bc37de74040c2ec4&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:58.272025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98282063-46fc-4f6b-a5d6-61915ed4453d None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:58.272025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98282063-46fc-4f6b-a5d6-61915ed4453d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:58.272025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98282063-46fc-4f6b-a5d6-61915ed4453d None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:58.272025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 473/946] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:58.274782 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98282063-46fc-4f6b-a5d6-61915ed4453d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:58.305860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febcee6f-4104-44ea-b232-b834d0ddb6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:58.306655 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcee6f-4104-44ea-b232-b834d0ddb6cc None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:58.306938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcee6f-4104-44ea-b232-b834d0ddb6cc None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:58.307205 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcee6f-4104-44ea-b232-b834d0ddb6cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:58.314017 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febcee6f-4104-44ea-b232-b834d0ddb6cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:58.315449 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febcee6f-4104-44ea-b232-b834d0ddb6cc None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-febcee6f-4104-44ea-b232-b834d0ddb6cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:58.329957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:58.330847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:58.331793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:58.331793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:58.331793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:58.331793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:58.331793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:58.331793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:58.331793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:58.331793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:58.331793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 474/947] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:58.332732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e0abd-e949-4c8d-9360-e77b916dc585 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:58.332732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e0abd-e949-4c8d-9360-e77b916dc585 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:58.332899 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e0abd-e949-4c8d-9360-e77b916dc585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:58.349395 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-d43e0abd-e949-4c8d-9360-e77b916dc585 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:58.350549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d43e0abd-e949-4c8d-9360-e77b916dc585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:58.380967 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98282063-46fc-4f6b-a5d6-61915ed4453d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:58.413848 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3874da2-7c3e-42f3-965c-1ffbd7f06bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:58.414512 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3874da2-7c3e-42f3-965c-1ffbd7f06bc5 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:58.414798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3874da2-7c3e-42f3-965c-1ffbd7f06bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:58.415070 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3874da2-7c3e-42f3-965c-1ffbd7f06bc5 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:58.415745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3874da2-7c3e-42f3-965c-1ffbd7f06bc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:58.417114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3874da2-7c3e-42f3-965c-1ffbd7f06bc5 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:58.432442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 474/948] 10.222.0.44 () {56 vars in 1087 bytes} [Mon May 9 04:13:58 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:13:58.432442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3874da2-7c3e-42f3-965c-1ffbd7f06bc5 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:58.432700 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 475/949] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:58.436476 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e0abd-e949-4c8d-9360-e77b916dc585 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:58.547509 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3874da2-7c3e-42f3-965c-1ffbd7f06bc5 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:58.566114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e0abd-e949-4c8d-9360-e77b916dc585 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:58.594745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f415f-0128-4dbd-b4a3-2ab1fde84dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:58.595755 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f415f-0128-4dbd-b4a3-2ab1fde84dc8 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:58.596077 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f415f-0128-4dbd-b4a3-2ab1fde84dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:58.596624 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f415f-0128-4dbd-b4a3-2ab1fde84dc8 None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:58.597439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f415f-0128-4dbd-b4a3-2ab1fde84dc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:58.597918 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f415f-0128-4dbd-b4a3-2ab1fde84dc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:58.600029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f415f-0128-4dbd-b4a3-2ab1fde84dc8 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:58.603247 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7658b7-2ae9-459f-869a-962a0f1867c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:58.604038 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7658b7-2ae9-459f-869a-962a0f1867c7 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:58.604442 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7658b7-2ae9-459f-869a-962a0f1867c7 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:58.604835 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7658b7-2ae9-459f-869a-962a0f1867c7 None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:58.605620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7658b7-2ae9-459f-869a-962a0f1867c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:58.607387 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7658b7-2ae9-459f-869a-962a0f1867c7 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:58.609082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: WARNING keystone.server.flask.common [None req-0e7658b7-2ae9-459f-869a-962a0f1867c7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 09 04:13:58.619286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 475/950] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:13:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:58.620238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f415f-0128-4dbd-b4a3-2ab1fde84dc8 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:58.630117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.sql.core [None req-0e7658b7-2ae9-459f-869a-962a0f1867c7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 04:13:58.630117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 04:13:58.630117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [parameters: {'id': '89ca199fc6544d8aa4087feaf38944fa', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 04:13:58.630117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79764) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 04:13:58.647162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-0e7658b7-2ae9-459f-869a-962a0f1867c7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 89ca199fc6544d8aa4087feaf38944fa.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 89ca199fc6544d8aa4087feaf38944fa. May 09 04:13:58.647162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:58.647162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 04:13:58.647162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 04:13:58.647162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 04:13:58.647162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 04:13:58.647162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 04:13:58.647162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application result = self._query(query) May 09 04:13:58.647162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 04:13:58.647162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application conn.query(q) May 09 04:13:58.647162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 04:13:58.647162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 04:13:58.647162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application result.read() May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 09 04:13:58.648068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return project_ref.to_dict() May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application next(self.gen) May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application yield resource May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application next(self.gen) May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self.session.rollback() May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self.force_reraise() May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise self.value May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application session.commit() May 09 04:13:58.649023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._prepare_impl() May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self.session.flush() May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._flush(objects) May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application compat.raise_( May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application flush_context.execute() May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 04:13:58.649918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rec.execute(self) May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 04:13:58.650825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application result = self._query(query) May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application conn.query(q) May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application result.read() May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 04:13:58.651636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [parameters: {'id': '89ca199fc6544d8aa4087feaf38944fa', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:58.654829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:58.658120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:58.661000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:58.661000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 09 04:13:58.661000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 09 04:13:58.661000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:58.661000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:58.661000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 09 04:13:58.661000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.Conflict( May 09 04:13:58.661000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 89ca199fc6544d8aa4087feaf38944fa. May 09 04:13:58.661000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:58.661000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 476/951] 10.222.0.44 () {60 vars in 1140 bytes} [Mon May 9 04:13:58 2022] POST /identity/v3/projects => generated 262 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 09 04:13:58.661000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7658b7-2ae9-459f-869a-962a0f1867c7 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:58.744098 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f415f-0128-4dbd-b4a3-2ab1fde84dc8 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:58.786852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68ab1b-cb04-47d4-93d4-ae3f3c0dd0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:58.788014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68ab1b-cb04-47d4-93d4-ae3f3c0dd0a3 None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:58.788297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68ab1b-cb04-47d4-93d4-ae3f3c0dd0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:58.788572 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68ab1b-cb04-47d4-93d4-ae3f3c0dd0a3 None admin] PATH_INFO: `/v3/projects/3322b00723974bd99e837d447bac22d1/users/ee0055c75b91471a99d9bb18248839ea/roles/7de5200fcd0943d9834b6e12861d5941` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:58.789474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7658b7-2ae9-459f-869a-962a0f1867c7 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:58.790792 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68ab1b-cb04-47d4-93d4-ae3f3c0dd0a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3322b00723974bd99e837d447bac22d1, user_id=ee0055c75b91471a99d9bb18248839ea, role_id=7de5200fcd0943d9834b6e12861d5941)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:58.793224 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68ab1b-cb04-47d4-93d4-ae3f3c0dd0a3 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:58.815689 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 476/952] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:13:58 2022] PUT /identity/v3/projects/3322b00723974bd99e837d447bac22d1/users/ee0055c75b91471a99d9bb18248839ea/roles/7de5200fcd0943d9834b6e12861d5941 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:13:58.834450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1d2c4-ff45-447a-804d-ce49f014a972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:58.835280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1d2c4-ff45-447a-804d-ce49f014a972 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:58.835557 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1d2c4-ff45-447a-804d-ce49f014a972 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:58.835867 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1d2c4-ff45-447a-804d-ce49f014a972 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:58.844075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1d2c4-ff45-447a-804d-ce49f014a972 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:58.845788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1d2c4-ff45-447a-804d-ce49f014a972 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-45f1d2c4-ff45-447a-804d-ce49f014a972 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:13:58.864574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:13:58.866662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:13:58.868793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:13:58.868793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:13:58.868793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:13:58.868793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:13:58.868793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:13:58.868793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:13:58.868793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:13:58.868793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:13:58.875934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 477/953] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:13:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:13:58.876235 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68ab1b-cb04-47d4-93d4-ae3f3c0dd0a3 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:58.995124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68ab1b-cb04-47d4-93d4-ae3f3c0dd0a3 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:59.034962 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afbbe8-7cbf-4d0a-ae15-47f7ae35b9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:59.035635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afbbe8-7cbf-4d0a-ae15-47f7ae35b9a0 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:59.035973 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afbbe8-7cbf-4d0a-ae15-47f7ae35b9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:59.036252 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afbbe8-7cbf-4d0a-ae15-47f7ae35b9a0 None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:59.037014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afbbe8-7cbf-4d0a-ae15-47f7ae35b9a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:13:59.037319 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afbbe8-7cbf-4d0a-ae15-47f7ae35b9a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:59.038741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afbbe8-7cbf-4d0a-ae15-47f7ae35b9a0 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:59.056588 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 477/954] 10.222.0.44 () {56 vars in 1112 bytes} [Mon May 9 04:13:58 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:13:59.623983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f01b4-9ea5-46eb-b8cc-b4fa5483dca4 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:59.624715 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f01b4-9ea5-46eb-b8cc-b4fa5483dca4 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:59.624715 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f01b4-9ea5-46eb-b8cc-b4fa5483dca4 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:59.628973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 478/955] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:13:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:13:59.634677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1355b-bd72-4be0-a976-0ae8a0d03dff None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:59.634971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1355b-bd72-4be0-a976-0ae8a0d03dff None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:59.635229 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1355b-bd72-4be0-a976-0ae8a0d03dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:59.654977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-70f1355b-bd72-4be0-a976-0ae8a0d03dff None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:13:59.656181 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-70f1355b-bd72-4be0-a976-0ae8a0d03dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:13:59.726886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 478/956] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:13:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:13:59.750116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f01b4-9ea5-46eb-b8cc-b4fa5483dca4 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:13:59.855825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f01b4-9ea5-46eb-b8cc-b4fa5483dca4 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:13:59.890744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4358fd-b568-4c55-9f26-0b876f8cfd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:13:59.891342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4358fd-b568-4c55-9f26-0b876f8cfd72 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:13:59.891626 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4358fd-b568-4c55-9f26-0b876f8cfd72 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:13:59.891922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4358fd-b568-4c55-9f26-0b876f8cfd72 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:13:59.892662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4358fd-b568-4c55-9f26-0b876f8cfd72 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:13:59.899849 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4358fd-b568-4c55-9f26-0b876f8cfd72 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:13:59.916780 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 479/957] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:13:59 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:14:00.774681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef64fa-3063-45c3-99e2-5f7a03354160 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:00.774681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef64fa-3063-45c3-99e2-5f7a03354160 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:00.774681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef64fa-3063-45c3-99e2-5f7a03354160 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:00.775715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 479/958] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:00.781542 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78489396-b50f-453b-b957-18a03a6fa001 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:00.782450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78489396-b50f-453b-b957-18a03a6fa001 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:00.782946 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78489396-b50f-453b-b957-18a03a6fa001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:00.799855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-78489396-b50f-453b-b957-18a03a6fa001 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:00.801706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-78489396-b50f-453b-b957-18a03a6fa001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:00.880885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 480/959] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:00.906669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef64fa-3063-45c3-99e2-5f7a03354160 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:01.012267 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef64fa-3063-45c3-99e2-5f7a03354160 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:01.050599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7350e5-d857-4fee-a0a2-ebfdb060acf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:01.051479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7350e5-d857-4fee-a0a2-ebfdb060acf2 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:01.051765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7350e5-d857-4fee-a0a2-ebfdb060acf2 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:01.052063 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7350e5-d857-4fee-a0a2-ebfdb060acf2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:01.052730 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7350e5-d857-4fee-a0a2-ebfdb060acf2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:01.054137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7350e5-d857-4fee-a0a2-ebfdb060acf2 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-9e7350e5-d857-4fee-a0a2-ebfdb060acf2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:01.067338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:01.068435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:01.069291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 480/960] 10.222.0.44 () {56 vars in 1126 bytes} [Mon May 9 04:14:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:01.077582 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78489396-b50f-453b-b957-18a03a6fa001 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:01.176891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78489396-b50f-453b-b957-18a03a6fa001 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:01.214080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584d5cd-5451-444b-8a90-5873dd31edb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:01.215107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584d5cd-5451-444b-8a90-5873dd31edb8 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:01.215620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584d5cd-5451-444b-8a90-5873dd31edb8 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:01.216150 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584d5cd-5451-444b-8a90-5873dd31edb8 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:01.217210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584d5cd-5451-444b-8a90-5873dd31edb8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:01.217751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584d5cd-5451-444b-8a90-5873dd31edb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:01.219312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584d5cd-5451-444b-8a90-5873dd31edb8 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:01.224859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 481/961] 10.222.0.44 () {56 vars in 1122 bytes} [Mon May 9 04:14:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:01.230645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7350e5-d857-4fee-a0a2-ebfdb060acf2 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:01.242041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23575f8c-bf25-46eb-b16d-a795a36536d0 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:01.242332 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23575f8c-bf25-46eb-b16d-a795a36536d0 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:01.242619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23575f8c-bf25-46eb-b16d-a795a36536d0 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:01.244708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 482/962] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:14:01.250867 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ae85d-e56c-4b22-89c2-24b5244a754d None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:01.251156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ae85d-e56c-4b22-89c2-24b5244a754d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:01.251435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ae85d-e56c-4b22-89c2-24b5244a754d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:01.268122 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-536ae85d-e56c-4b22-89c2-24b5244a754d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:01.269231 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-536ae85d-e56c-4b22-89c2-24b5244a754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:01.344012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 483/963] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:14:01.350580 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7350e5-d857-4fee-a0a2-ebfdb060acf2 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:01.359712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f079f-fe8a-44c3-b767-ba7743b42524 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:01.360067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f079f-fe8a-44c3-b767-ba7743b42524 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:01.360354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f079f-fe8a-44c3-b767-ba7743b42524 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:01.361835 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 484/964] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:14:01.366207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f079f-fe8a-44c3-b767-ba7743b42524 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:01.388111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3032e-7028-4306-ab8d-02076c455045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:01.388734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3032e-7028-4306-ab8d-02076c455045 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:01.388985 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3032e-7028-4306-ab8d-02076c455045 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:01.389259 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3032e-7028-4306-ab8d-02076c455045 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:01.390021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3032e-7028-4306-ab8d-02076c455045 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:01.390271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3032e-7028-4306-ab8d-02076c455045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:01.391636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3032e-7028-4306-ab8d-02076c455045 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:01.401110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 481/965] 10.222.0.44 () {56 vars in 1122 bytes} [Mon May 9 04:14:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:01.401110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8e73e-1811-45b8-b5a6-06641aca0458 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:01.401110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8e73e-1811-45b8-b5a6-06641aca0458 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:01.401110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8e73e-1811-45b8-b5a6-06641aca0458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:01.415884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-edc8e73e-1811-45b8-b5a6-06641aca0458 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:01.417071 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-edc8e73e-1811-45b8-b5a6-06641aca0458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:01.469894 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f079f-fe8a-44c3-b767-ba7743b42524 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:01.488634 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 482/966] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:01.489721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8e73e-1811-45b8-b5a6-06641aca0458 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:01.515358 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bfb6a-e595-44d2-b0ff-03d4af5c3c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:01.516937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bfb6a-e595-44d2-b0ff-03d4af5c3c6d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:01.517695 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bfb6a-e595-44d2-b0ff-03d4af5c3c6d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:01.518204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bfb6a-e595-44d2-b0ff-03d4af5c3c6d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:01.531914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bfb6a-e595-44d2-b0ff-03d4af5c3c6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-056bfb6a-e595-44d2-b0ff-03d4af5c3c6d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:14:01.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:01.543215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:01.544674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:01.544674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:01.544674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:01.544674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:01.544674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:14:01.544674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:14:01.544674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:14:01.544674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:14:01.544674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:14:01.544674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:14:01.544674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 04:14:01.544674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:01.546185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 485/967] 10.222.0.44 () {56 vars in 1120 bytes} [Mon May 9 04:14:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:01.546185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bfb6a-e595-44d2-b0ff-03d4af5c3c6d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:01.591459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8e73e-1811-45b8-b5a6-06641aca0458 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:01.624803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6c635-796e-4312-aa5c-bb0fcb998e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:01.625467 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6c635-796e-4312-aa5c-bb0fcb998e89 None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:01.625754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6c635-796e-4312-aa5c-bb0fcb998e89 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:01.626033 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6c635-796e-4312-aa5c-bb0fcb998e89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:01.626776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6c635-796e-4312-aa5c-bb0fcb998e89 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:01.628169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6c635-796e-4312-aa5c-bb0fcb998e89 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:01.644344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bfb6a-e595-44d2-b0ff-03d4af5c3c6d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:01.651940 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 483/968] 10.222.0.44 () {60 vars in 1144 bytes} [Mon May 9 04:14:01 2022] POST /identity/v3/endpoints => generated 349 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:14:01.651940 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6c635-796e-4312-aa5c-bb0fcb998e89 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:01.709629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef887c3-d600-4f9d-b63c-8d61d24aaae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:01.710615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef887c3-d600-4f9d-b63c-8d61d24aaae1 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:01.711044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef887c3-d600-4f9d-b63c-8d61d24aaae1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:01.711496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef887c3-d600-4f9d-b63c-8d61d24aaae1 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:01.712391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef887c3-d600-4f9d-b63c-8d61d24aaae1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:01.714075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef887c3-d600-4f9d-b63c-8d61d24aaae1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:01.730818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.sql.core [None req-7ef887c3-d600-4f9d-b63c-8d61d24aaae1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 04:14:01.730818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 04:14:01.730818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [parameters: {'user_id': 'ae17a97d40444371863e90aa95831f71', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 04:14:01.730818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=79764) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-7ef887c3-d600-4f9d-b63c-8d61d24aaae1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application result = self._query(query) May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application conn.query(q) May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application result.read() May 09 04:14:01.739814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 04:14:01.740994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application next(self.gen) May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application yield resource May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application next(self.gen) May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self.session.rollback() May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self.force_reraise() May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise self.value May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application session.commit() May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 04:14:01.742188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._prepare_impl() May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self.session.flush() May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._flush(objects) May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application compat.raise_( May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application flush_context.execute() May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rec.execute(self) May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 04:14:01.743116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application result = connection._execute_20( May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 04:14:01.744129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application result = self._query(query) May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application conn.query(q) May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application result.read() May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 04:14:01.744969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ae17a97d40444371863e90aa95831f71', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:01.745827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 09 04:14:01.746732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 09 04:14:01.747920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 09 04:14:01.747920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 09 04:14:01.747920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 04:14:01.747920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 04:14:01.747920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 04:14:01.747920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:01.760402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 486/969] 10.222.0.44 () {60 vars in 1131 bytes} [Mon May 9 04:14:01 2022] POST /identity/v3/users => generated 160 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:01.764567 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6c635-796e-4312-aa5c-bb0fcb998e89 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:01.766070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef887c3-d600-4f9d-b63c-8d61d24aaae1 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:01.805970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79ae4a-3805-462c-9cf8-a0ae9f25c17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:01.807878 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79ae4a-3805-462c-9cf8-a0ae9f25c17d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:01.808347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79ae4a-3805-462c-9cf8-a0ae9f25c17d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:01.808861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79ae4a-3805-462c-9cf8-a0ae9f25c17d None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:01.809829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79ae4a-3805-462c-9cf8-a0ae9f25c17d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:01.810350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79ae4a-3805-462c-9cf8-a0ae9f25c17d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:01.813092 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79ae4a-3805-462c-9cf8-a0ae9f25c17d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:01.824863 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 484/970] 10.222.0.44 () {56 vars in 1115 bytes} [Mon May 9 04:14:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:01.831940 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79ae4a-3805-462c-9cf8-a0ae9f25c17d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:01.897931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef887c3-d600-4f9d-b63c-8d61d24aaae1 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:01.948900 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79ae4a-3805-462c-9cf8-a0ae9f25c17d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:01.949127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252d748-29b2-4ed8-a561-bc831cfdfa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:01.949765 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252d748-29b2-4ed8-a561-bc831cfdfa7f None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:01.950560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252d748-29b2-4ed8-a561-bc831cfdfa7f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:01.951000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252d748-29b2-4ed8-a561-bc831cfdfa7f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:01.960975 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252d748-29b2-4ed8-a561-bc831cfdfa7f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:01.963150 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252d748-29b2-4ed8-a561-bc831cfdfa7f None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-4252d748-29b2-4ed8-a561-bc831cfdfa7f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:14:01.982078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:01.983207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:01.984245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:01.984245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:01.984245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:14:01.984245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:14:01.984245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:14:01.984245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:14:01.984245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:14:01.984245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:14:01.984245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 04:14:01.984245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:01.984245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 487/971] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:14:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:01.987612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d1ea3-18e0-46a8-b2bd-4c4f13f79bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:01.988707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d1ea3-18e0-46a8-b2bd-4c4f13f79bc7 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:01.988848 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252d748-29b2-4ed8-a561-bc831cfdfa7f None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:01.989441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d1ea3-18e0-46a8-b2bd-4c4f13f79bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:01.989921 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d1ea3-18e0-46a8-b2bd-4c4f13f79bc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:02.001390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d1ea3-18e0-46a8-b2bd-4c4f13f79bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:02.003293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d1ea3-18e0-46a8-b2bd-4c4f13f79bc7 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-eb4d1ea3-18e0-46a8-b2bd-4c4f13f79bc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:14:02.027091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:02.028746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:02.029974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:14:02.029974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:14:02.029974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:02.029974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:02.032387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 485/972] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:02.040022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d1ea3-18e0-46a8-b2bd-4c4f13f79bc7 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:02.105505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252d748-29b2-4ed8-a561-bc831cfdfa7f None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:02.140505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640f639-3cb2-4a13-8a4e-b4ded1567546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:02.141279 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640f639-3cb2-4a13-8a4e-b4ded1567546 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:02.141459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640f639-3cb2-4a13-8a4e-b4ded1567546 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:02.141744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640f639-3cb2-4a13-8a4e-b4ded1567546 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:02.142614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640f639-3cb2-4a13-8a4e-b4ded1567546 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:02.142857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640f639-3cb2-4a13-8a4e-b4ded1567546 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:02.144299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640f639-3cb2-4a13-8a4e-b4ded1567546 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:02.150461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d1ea3-18e0-46a8-b2bd-4c4f13f79bc7 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:02.191947 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 488/973] 10.222.0.44 () {56 vars in 1101 bytes} [Mon May 9 04:14:01 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:02.192155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb918b-3f9f-4e33-840e-0120a74cd109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:02.192155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb918b-3f9f-4e33-840e-0120a74cd109 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:02.192155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb918b-3f9f-4e33-840e-0120a74cd109 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:02.192155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb918b-3f9f-4e33-840e-0120a74cd109 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:02.192477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb918b-3f9f-4e33-840e-0120a74cd109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:02.192477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb918b-3f9f-4e33-840e-0120a74cd109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:02.192477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb918b-3f9f-4e33-840e-0120a74cd109 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:02.207181 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80283f-b8ee-4084-91a0-4fb4c606c1bf None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:02.207493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80283f-b8ee-4084-91a0-4fb4c606c1bf None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:02.207747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80283f-b8ee-4084-91a0-4fb4c606c1bf None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:02.210727 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 486/974] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:02.213401 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb918b-3f9f-4e33-840e-0120a74cd109 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:02.213560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 489/975] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:14:02.214982 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430303d7-2b50-463d-a3f7-86eb1e6f7cad None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:02.215278 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430303d7-2b50-463d-a3f7-86eb1e6f7cad None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:02.215597 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430303d7-2b50-463d-a3f7-86eb1e6f7cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:02.239417 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-430303d7-2b50-463d-a3f7-86eb1e6f7cad None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:02.240990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-430303d7-2b50-463d-a3f7-86eb1e6f7cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:02.330250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 490/976] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:02.333722 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb918b-3f9f-4e33-840e-0120a74cd109 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:02.352560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430303d7-2b50-463d-a3f7-86eb1e6f7cad None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:02.381454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a839a3-5eb5-4c5d-866c-b48a85eecb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:02.382471 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a839a3-5eb5-4c5d-866c-b48a85eecb74 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:02.382916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a839a3-5eb5-4c5d-866c-b48a85eecb74 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:02.383203 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a839a3-5eb5-4c5d-866c-b48a85eecb74 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:02.394747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a839a3-5eb5-4c5d-866c-b48a85eecb74 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:02.396731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a839a3-5eb5-4c5d-866c-b48a85eecb74 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-09a839a3-5eb5-4c5d-866c-b48a85eecb74 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:14:02.413875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:02.414857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:02.415853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:02.415853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:02.415853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:14:02.415853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:14:02.415853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:14:02.415853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:14:02.415853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:14:02.415853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:14:02.415853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 04:14:02.415853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:02.415853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 487/977] 10.222.0.44 () {56 vars in 1117 bytes} [Mon May 9 04:14:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:02.421355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a839a3-5eb5-4c5d-866c-b48a85eecb74 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:02.466636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430303d7-2b50-463d-a3f7-86eb1e6f7cad None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:02.510783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5951cd9-382c-43a8-839e-00efc7ccda62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:02.512177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5951cd9-382c-43a8-839e-00efc7ccda62 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:02.513142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5951cd9-382c-43a8-839e-00efc7ccda62 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:02.513321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5951cd9-382c-43a8-839e-00efc7ccda62 None admin] PATH_INFO: `/v3/services/compute` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:02.513940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5951cd9-382c-43a8-839e-00efc7ccda62 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:02.515305 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5951cd9-382c-43a8-839e-00efc7ccda62 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-e5951cd9-382c-43a8-839e-00efc7ccda62 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:02.527696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:02.529191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 04:14:02.530153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:02.530921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 491/978] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:14:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:02.537662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5951cd9-382c-43a8-839e-00efc7ccda62 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:02.553115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a839a3-5eb5-4c5d-866c-b48a85eecb74 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:02.587625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628b0eb-58b5-4697-ba80-2f4513ca0068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:02.588325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628b0eb-58b5-4697-ba80-2f4513ca0068 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:02.588605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628b0eb-58b5-4697-ba80-2f4513ca0068 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:02.588880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628b0eb-58b5-4697-ba80-2f4513ca0068 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:02.589708 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628b0eb-58b5-4697-ba80-2f4513ca0068 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:02.590017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628b0eb-58b5-4697-ba80-2f4513ca0068 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:02.591611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628b0eb-58b5-4697-ba80-2f4513ca0068 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:02.609404 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 488/979] 10.222.0.44 () {56 vars in 1149 bytes} [Mon May 9 04:14:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:02.614879 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628b0eb-58b5-4697-ba80-2f4513ca0068 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:02.640431 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5951cd9-382c-43a8-839e-00efc7ccda62 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:02.676737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399b28d-9559-4c78-a2b1-72c96d093a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:02.677774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399b28d-9559-4c78-a2b1-72c96d093a50 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:02.678674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399b28d-9559-4c78-a2b1-72c96d093a50 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:02.679138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399b28d-9559-4c78-a2b1-72c96d093a50 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:02.680316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399b28d-9559-4c78-a2b1-72c96d093a50 None admin] RBAC: Adding query filter params (name=compute) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:02.680966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399b28d-9559-4c78-a2b1-72c96d093a50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:02.683244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399b28d-9559-4c78-a2b1-72c96d093a50 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:02.689863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 492/980] 10.222.0.44 () {56 vars in 1112 bytes} [Mon May 9 04:14:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:02.695353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399b28d-9559-4c78-a2b1-72c96d093a50 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:02.714494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628b0eb-58b5-4697-ba80-2f4513ca0068 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:02.754121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7aef5-96fa-4671-a5c5-d77122d5a5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:02.755093 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7aef5-96fa-4671-a5c5-d77122d5a5d7 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:02.755558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7aef5-96fa-4671-a5c5-d77122d5a5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:02.756042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7aef5-96fa-4671-a5c5-d77122d5a5d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:02.796125 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399b28d-9559-4c78-a2b1-72c96d093a50 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:02.821106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7aef5-96fa-4671-a5c5-d77122d5a5d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:02.822557 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7aef5-96fa-4671-a5c5-d77122d5a5d7 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:02.828724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af58b7f-342f-43e0-844d-7e06e6d176df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:02.829652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af58b7f-342f-43e0-844d-7e06e6d176df None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:02.829961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af58b7f-342f-43e0-844d-7e06e6d176df None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:02.830233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af58b7f-342f-43e0-844d-7e06e6d176df None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:02.830929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af58b7f-342f-43e0-844d-7e06e6d176df None admin] RBAC: Adding query filter params (type=compute) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:02.831229 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af58b7f-342f-43e0-844d-7e06e6d176df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:02.832595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af58b7f-342f-43e0-844d-7e06e6d176df None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:02.837387 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 493/981] 10.222.0.44 () {56 vars in 1112 bytes} [Mon May 9 04:14:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:02.843338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af58b7f-342f-43e0-844d-7e06e6d176df None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:02.927147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 489/982] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:14:02 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 04:14:02.928172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7aef5-96fa-4671-a5c5-d77122d5a5d7 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:02.939216 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af58b7f-342f-43e0-844d-7e06e6d176df None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:02.974100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1508a-303a-4c81-9ea0-9de8a32badad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:02.974762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1508a-303a-4c81-9ea0-9de8a32badad None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:02.975041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1508a-303a-4c81-9ea0-9de8a32badad None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:02.975313 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1508a-303a-4c81-9ea0-9de8a32badad None admin] PATH_INFO: `/v3/endpoints` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:02.976144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1508a-303a-4c81-9ea0-9de8a32badad None admin] RBAC: Adding query filter params (interface=public, service_id=d123680c05264f5a8544cc14201dcfaa, region_id=RegionOne) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:02.976454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1508a-303a-4c81-9ea0-9de8a32badad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:02.977816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1508a-303a-4c81-9ea0-9de8a32badad None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:02.984103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 494/983] 10.222.0.44 () {56 vars in 1251 bytes} [Mon May 9 04:14:02 2022] GET /identity/v3/endpoints?service_id=d123680c05264f5a8544cc14201dcfaa&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:02.989203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1508a-303a-4c81-9ea0-9de8a32badad None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:03.041997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7aef5-96fa-4671-a5c5-d77122d5a5d7 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:03.076106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b5d1c-3dd5-407e-8b3c-1b0acc46d4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:03.076980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b5d1c-3dd5-407e-8b3c-1b0acc46d4f6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:03.077434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b5d1c-3dd5-407e-8b3c-1b0acc46d4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:03.077716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b5d1c-3dd5-407e-8b3c-1b0acc46d4f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:03.086326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1508a-303a-4c81-9ea0-9de8a32badad None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:03.089084 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b5d1c-3dd5-407e-8b3c-1b0acc46d4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:03.090322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b5d1c-3dd5-407e-8b3c-1b0acc46d4f6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-9e9b5d1c-3dd5-407e-8b3c-1b0acc46d4f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:14:03.108789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:03.109788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:03.110682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:14:03.110682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:14:03.110682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:03.110682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:03.110682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 490/984] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:03.115305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b5d1c-3dd5-407e-8b3c-1b0acc46d4f6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:03.122638 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0046c-6dff-4ea3-944c-0c2f2ffd7ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:03.123309 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0046c-6dff-4ea3-944c-0c2f2ffd7ea6 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:03.123587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0046c-6dff-4ea3-944c-0c2f2ffd7ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:03.123862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0046c-6dff-4ea3-944c-0c2f2ffd7ea6 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:03.124571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0046c-6dff-4ea3-944c-0c2f2ffd7ea6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:03.125941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0046c-6dff-4ea3-944c-0c2f2ffd7ea6 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:03.140163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 495/985] 10.222.0.44 () {56 vars in 1087 bytes} [Mon May 9 04:14:02 2022] GET /identity/v3/services => generated 1727 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:14:03.227568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b5d1c-3dd5-407e-8b3c-1b0acc46d4f6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:03.261786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd3f3c-3d3d-480a-976c-b23f2e9470af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:03.263118 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd3f3c-3d3d-480a-976c-b23f2e9470af None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:03.263402 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd3f3c-3d3d-480a-976c-b23f2e9470af None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:03.263692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd3f3c-3d3d-480a-976c-b23f2e9470af None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:03.264610 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd3f3c-3d3d-480a-976c-b23f2e9470af None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:03.264921 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd3f3c-3d3d-480a-976c-b23f2e9470af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:03.266339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd3f3c-3d3d-480a-976c-b23f2e9470af None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:03.284687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 491/986] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:03.291807 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0046c-6dff-4ea3-944c-0c2f2ffd7ea6 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:03.397251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0046c-6dff-4ea3-944c-0c2f2ffd7ea6 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:03.429007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d1120-69d5-489b-8915-a8fa5c0e6221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:03.429802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1120-69d5-489b-8915-a8fa5c0e6221 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:03.430085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1120-69d5-489b-8915-a8fa5c0e6221 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:03.430360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1120-69d5-489b-8915-a8fa5c0e6221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:03.437470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d1120-69d5-489b-8915-a8fa5c0e6221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:03.438903 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d1120-69d5-489b-8915-a8fa5c0e6221 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-973d1120-69d5-489b-8915-a8fa5c0e6221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:14:03.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:03.454194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:03.455099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:14:03.455099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:14:03.455099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:14:03.455099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:14:03.455099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:14:03.455099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:14:03.455099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:14:03.455099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:03.455099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 496/987] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:14:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:03.460409 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd3f3c-3d3d-480a-976c-b23f2e9470af None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:03.556196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd3f3c-3d3d-480a-976c-b23f2e9470af None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:03.589398 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8f4bd-19ab-4ff0-8d5d-c60b24fe7a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:03.590061 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8f4bd-19ab-4ff0-8d5d-c60b24fe7a2f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:03.590345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8f4bd-19ab-4ff0-8d5d-c60b24fe7a2f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:03.591228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8f4bd-19ab-4ff0-8d5d-c60b24fe7a2f None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:03.592186 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8f4bd-19ab-4ff0-8d5d-c60b24fe7a2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:03.592497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8f4bd-19ab-4ff0-8d5d-c60b24fe7a2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:03.593929 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8f4bd-19ab-4ff0-8d5d-c60b24fe7a2f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:03.607198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 492/988] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:14:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:03.612553 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d1120-69d5-489b-8915-a8fa5c0e6221 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:03.708708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d1120-69d5-489b-8915-a8fa5c0e6221 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:03.742161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578e264-676e-4950-9012-08e4517f3bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:03.743149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578e264-676e-4950-9012-08e4517f3bbb None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:03.743633 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578e264-676e-4950-9012-08e4517f3bbb None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:03.744132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578e264-676e-4950-9012-08e4517f3bbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:03.745179 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578e264-676e-4950-9012-08e4517f3bbb None admin] RBAC: Adding query filter params (role.id=7de5200fcd0943d9834b6e12861d5941, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=ee0055c75b91471a99d9bb18248839ea) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:03.745647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578e264-676e-4950-9012-08e4517f3bbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:03.747739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578e264-676e-4950-9012-08e4517f3bbb None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:03.754429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 497/989] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:14:03 2022] GET /identity/v3/role_assignments?user.id=ee0055c75b91471a99d9bb18248839ea&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=7de5200fcd0943d9834b6e12861d5941 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:03.994382 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff56139-5dd9-4c25-afca-1bfab244bd8e None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:03.994932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff56139-5dd9-4c25-afca-1bfab244bd8e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:03.994932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff56139-5dd9-4c25-afca-1bfab244bd8e None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:03.996315 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 493/990] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:04.001942 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa50c6-2b21-433a-b7ba-73013f944a36 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:04.002390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa50c6-2b21-433a-b7ba-73013f944a36 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:04.002484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa50c6-2b21-433a-b7ba-73013f944a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:04.018605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-ceaa50c6-2b21-433a-b7ba-73013f944a36 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:04.019757 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ceaa50c6-2b21-433a-b7ba-73013f944a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:04.082753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 498/991] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:04 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:04.105141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff56139-5dd9-4c25-afca-1bfab244bd8e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:04.199553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff56139-5dd9-4c25-afca-1bfab244bd8e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:04.232783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9e96a-d44b-447c-9a74-1732c59da9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:04.233580 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9e96a-d44b-447c-9a74-1732c59da9a6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:04.233856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9e96a-d44b-447c-9a74-1732c59da9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:04.234123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9e96a-d44b-447c-9a74-1732c59da9a6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:04.234765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9e96a-d44b-447c-9a74-1732c59da9a6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:04.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9e96a-d44b-447c-9a74-1732c59da9a6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-ada9e96a-d44b-447c-9a74-1732c59da9a6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:04.248462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:04.249445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:14:04.250288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:04.251983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 494/992] 10.222.0.44 () {56 vars in 1126 bytes} [Mon May 9 04:14:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:04.255908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa50c6-2b21-433a-b7ba-73013f944a36 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:04.352680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa50c6-2b21-433a-b7ba-73013f944a36 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:04.384753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56927f1d-7c87-4312-bc91-49814814a8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:04.385421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927f1d-7c87-4312-bc91-49814814a8ab None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:04.385703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927f1d-7c87-4312-bc91-49814814a8ab None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:04.385976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927f1d-7c87-4312-bc91-49814814a8ab None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:04.386676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56927f1d-7c87-4312-bc91-49814814a8ab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:04.386980 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56927f1d-7c87-4312-bc91-49814814a8ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:04.389206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56927f1d-7c87-4312-bc91-49814814a8ab None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:04.398379 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 499/993] 10.222.0.44 () {56 vars in 1122 bytes} [Mon May 9 04:14:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:04.399710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9e96a-d44b-447c-9a74-1732c59da9a6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:04.494664 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9e96a-d44b-447c-9a74-1732c59da9a6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:04.527204 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51127cf-dbde-47a6-9707-2f4da15f9f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:04.527798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51127cf-dbde-47a6-9707-2f4da15f9f37 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:04.528170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51127cf-dbde-47a6-9707-2f4da15f9f37 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:04.528522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51127cf-dbde-47a6-9707-2f4da15f9f37 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:04.529233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51127cf-dbde-47a6-9707-2f4da15f9f37 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:04.529541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51127cf-dbde-47a6-9707-2f4da15f9f37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:04.530904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51127cf-dbde-47a6-9707-2f4da15f9f37 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:04.536150 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 495/994] 10.222.0.44 () {56 vars in 1122 bytes} [Mon May 9 04:14:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:04.541769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56927f1d-7c87-4312-bc91-49814814a8ab None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:04.637108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56927f1d-7c87-4312-bc91-49814814a8ab None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:04.669274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed0277-2107-466a-bb66-c2958c57c796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:04.669931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed0277-2107-466a-bb66-c2958c57c796 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:04.670213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed0277-2107-466a-bb66-c2958c57c796 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:04.670487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed0277-2107-466a-bb66-c2958c57c796 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:04.671282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed0277-2107-466a-bb66-c2958c57c796 None admin] RBAC: Adding query filter params (interface=admin, service_id=99d9c58a99484ca8bc37de74040c2ec4, region_id=RegionOne) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:04.671730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed0277-2107-466a-bb66-c2958c57c796 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:04.673125 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed0277-2107-466a-bb66-c2958c57c796 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:04.682358 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 500/995] 10.222.0.44 () {56 vars in 1249 bytes} [Mon May 9 04:14:04 2022] GET /identity/v3/endpoints?service_id=99d9c58a99484ca8bc37de74040c2ec4&interface=admin®ion_id=RegionOne => generated 193 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:04.683491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51127cf-dbde-47a6-9707-2f4da15f9f37 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:04.781990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51127cf-dbde-47a6-9707-2f4da15f9f37 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:04.814135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee616d-125d-446e-860c-85d6a4065c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:04.814801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee616d-125d-446e-860c-85d6a4065c6c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:04.815086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee616d-125d-446e-860c-85d6a4065c6c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:04.815440 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee616d-125d-446e-860c-85d6a4065c6c None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:04.816153 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee616d-125d-446e-860c-85d6a4065c6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:04.817510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee616d-125d-446e-860c-85d6a4065c6c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:04.827925 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 496/996] 10.222.0.44 () {56 vars in 1087 bytes} [Mon May 9 04:14:04 2022] GET /identity/v3/services => generated 1727 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:14:05.406434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbd315-5e75-4ae7-8a6a-22b1d0d8c9ee None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:05.406953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbd315-5e75-4ae7-8a6a-22b1d0d8c9ee None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:05.407031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbd315-5e75-4ae7-8a6a-22b1d0d8c9ee None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:05.408422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 501/997] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:05.413793 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714edc02-6e3f-4a7d-8563-0b216b80fd6d None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:05.414207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714edc02-6e3f-4a7d-8563-0b216b80fd6d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:05.414381 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714edc02-6e3f-4a7d-8563-0b216b80fd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:05.431148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-714edc02-6e3f-4a7d-8563-0b216b80fd6d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:05.432327 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-714edc02-6e3f-4a7d-8563-0b216b80fd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:05.495097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 497/998] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:05.518395 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbd315-5e75-4ae7-8a6a-22b1d0d8c9ee None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:05.615325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbd315-5e75-4ae7-8a6a-22b1d0d8c9ee None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:05.647483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e0aed-74f9-465c-9afa-c624b986a352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:05.648300 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e0aed-74f9-465c-9afa-c624b986a352 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:05.648581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e0aed-74f9-465c-9afa-c624b986a352 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:05.648851 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e0aed-74f9-465c-9afa-c624b986a352 None admin] PATH_INFO: `/v3/services/compute` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:05.649493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e0aed-74f9-465c-9afa-c624b986a352 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:05.650844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e0aed-74f9-465c-9afa-c624b986a352 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-4a4e0aed-74f9-465c-9afa-c624b986a352 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:05.662472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:05.663453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 04:14:05.664746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:05.668212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 502/999] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:14:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:05.670167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714edc02-6e3f-4a7d-8563-0b216b80fd6d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:05.764898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714edc02-6e3f-4a7d-8563-0b216b80fd6d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:05.796747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883cadd-3006-4d42-b2d1-ba49d4fb0d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:05.797404 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883cadd-3006-4d42-b2d1-ba49d4fb0d4e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:05.797683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883cadd-3006-4d42-b2d1-ba49d4fb0d4e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:05.797953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883cadd-3006-4d42-b2d1-ba49d4fb0d4e None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:05.798646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883cadd-3006-4d42-b2d1-ba49d4fb0d4e None admin] RBAC: Adding query filter params (name=compute) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:05.798944 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883cadd-3006-4d42-b2d1-ba49d4fb0d4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:05.800311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883cadd-3006-4d42-b2d1-ba49d4fb0d4e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:05.809959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 498/1000] 10.222.0.44 () {56 vars in 1112 bytes} [Mon May 9 04:14:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:05.811369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e0aed-74f9-465c-9afa-c624b986a352 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:05.916282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e0aed-74f9-465c-9afa-c624b986a352 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:05.949031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0722a-5847-4093-8319-6109b9375fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:05.949694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0722a-5847-4093-8319-6109b9375fcf None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:05.950006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0722a-5847-4093-8319-6109b9375fcf None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:05.950281 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0722a-5847-4093-8319-6109b9375fcf None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:05.950990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0722a-5847-4093-8319-6109b9375fcf None admin] RBAC: Adding query filter params (type=compute) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:05.951290 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0722a-5847-4093-8319-6109b9375fcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:05.952682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0722a-5847-4093-8319-6109b9375fcf None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:05.957819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 503/1001] 10.222.0.44 () {56 vars in 1112 bytes} [Mon May 9 04:14:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:05.963377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883cadd-3006-4d42-b2d1-ba49d4fb0d4e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:06.010918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7962bb-0814-4493-b2d8-2f1af09bdebd None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:06.011261 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7962bb-0814-4493-b2d8-2f1af09bdebd None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:06.011477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7962bb-0814-4493-b2d8-2f1af09bdebd None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:06.013076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 504/1002] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:14:06.017976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfc0aa-7c63-4d78-940f-c5705a8bc863 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:06.018264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfc0aa-7c63-4d78-940f-c5705a8bc863 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:06.018543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfc0aa-7c63-4d78-940f-c5705a8bc863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:06.035158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e3bfc0aa-7c63-4d78-940f-c5705a8bc863 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:06.036343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e3bfc0aa-7c63-4d78-940f-c5705a8bc863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:06.063653 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883cadd-3006-4d42-b2d1-ba49d4fb0d4e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:06.098342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e91290-660f-46bb-88d0-5c78e80a65e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:06.099002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e91290-660f-46bb-88d0-5c78e80a65e8 None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:06.099283 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e91290-660f-46bb-88d0-5c78e80a65e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:06.099553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e91290-660f-46bb-88d0-5c78e80a65e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:06.100330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e91290-660f-46bb-88d0-5c78e80a65e8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:06.101682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e91290-660f-46bb-88d0-5c78e80a65e8 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:06.103695 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 505/1003] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:14:06.123396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 499/1004] 10.222.0.44 () {60 vars in 1144 bytes} [Mon May 9 04:14:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:14:06.126829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfc0aa-7c63-4d78-940f-c5705a8bc863 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:06.228680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfc0aa-7c63-4d78-940f-c5705a8bc863 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:06.272654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020b564-ddf1-414d-9157-582ffa03b96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:06.273466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020b564-ddf1-414d-9157-582ffa03b96b None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:06.273751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020b564-ddf1-414d-9157-582ffa03b96b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:06.274028 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020b564-ddf1-414d-9157-582ffa03b96b None admin] PATH_INFO: `/v3/services/image` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:06.274690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020b564-ddf1-414d-9157-582ffa03b96b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:06.276103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020b564-ddf1-414d-9157-582ffa03b96b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-3020b564-ddf1-414d-9157-582ffa03b96b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:06.288334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:06.289408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:06.295441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 506/1005] 10.222.0.44 () {56 vars in 1105 bytes} [Mon May 9 04:14:06 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:06.297907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e91290-660f-46bb-88d0-5c78e80a65e8 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:06.398896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e91290-660f-46bb-88d0-5c78e80a65e8 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:06.433115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152b6c4-66c8-46ef-ac1e-f5d235a7e952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:06.433887 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152b6c4-66c8-46ef-ac1e-f5d235a7e952 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:06.434065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152b6c4-66c8-46ef-ac1e-f5d235a7e952 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:06.434982 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152b6c4-66c8-46ef-ac1e-f5d235a7e952 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:06.435709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3152b6c4-66c8-46ef-ac1e-f5d235a7e952 None admin] RBAC: Adding query filter params (name=image) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:06.436048 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3152b6c4-66c8-46ef-ac1e-f5d235a7e952 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:06.437610 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3152b6c4-66c8-46ef-ac1e-f5d235a7e952 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:06.443547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 500/1006] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:06.448413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020b564-ddf1-414d-9157-582ffa03b96b None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:06.544459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020b564-ddf1-414d-9157-582ffa03b96b None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:06.575880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4a5cb-bf71-45e4-9d21-acba1abf4e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:06.576572 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4a5cb-bf71-45e4-9d21-acba1abf4e8c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:06.576851 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4a5cb-bf71-45e4-9d21-acba1abf4e8c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:06.577117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4a5cb-bf71-45e4-9d21-acba1abf4e8c None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:06.577811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4a5cb-bf71-45e4-9d21-acba1abf4e8c None admin] RBAC: Adding query filter params (type=image) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:06.578109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4a5cb-bf71-45e4-9d21-acba1abf4e8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:06.579731 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc4a5cb-bf71-45e4-9d21-acba1abf4e8c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:06.584252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 507/1007] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:06 2022] GET /identity/v3/services?type=image => generated 122 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:07.102223 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1b909-ca4b-4a1a-aa23-0919d90d2efd None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:07.102603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1b909-ca4b-4a1a-aa23-0919d90d2efd None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:07.102800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1b909-ca4b-4a1a-aa23-0919d90d2efd None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:07.104331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 501/1008] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:07.109382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f62eb-5e9e-4538-8d36-fc3d46e5ba02 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:07.109672 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f62eb-5e9e-4538-8d36-fc3d46e5ba02 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:07.110079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f62eb-5e9e-4538-8d36-fc3d46e5ba02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:07.126347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-a75f62eb-5e9e-4538-8d36-fc3d46e5ba02 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:07.127518 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a75f62eb-5e9e-4538-8d36-fc3d46e5ba02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:07.190329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 508/1009] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:07 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:07.213248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1b909-ca4b-4a1a-aa23-0919d90d2efd None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:07.308368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1b909-ca4b-4a1a-aa23-0919d90d2efd None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:07.341993 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47831f33-60ca-43a5-92bf-432bbec528ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:07.343221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47831f33-60ca-43a5-92bf-432bbec528ff None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:07.343707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47831f33-60ca-43a5-92bf-432bbec528ff None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:07.344204 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47831f33-60ca-43a5-92bf-432bbec528ff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:07.345081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47831f33-60ca-43a5-92bf-432bbec528ff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:07.346715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47831f33-60ca-43a5-92bf-432bbec528ff None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-47831f33-60ca-43a5-92bf-432bbec528ff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:07.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:07.361013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 04:14:07.361893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:07.364042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 502/1010] 10.222.0.44 () {56 vars in 1126 bytes} [Mon May 9 04:14:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:07.367664 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f62eb-5e9e-4538-8d36-fc3d46e5ba02 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:07.463524 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f62eb-5e9e-4538-8d36-fc3d46e5ba02 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:07.495685 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a0893-b340-43eb-a9ea-b2636090cd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:07.496384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a0893-b340-43eb-a9ea-b2636090cd63 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:07.496665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a0893-b340-43eb-a9ea-b2636090cd63 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:07.497453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a0893-b340-43eb-a9ea-b2636090cd63 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:07.498262 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a0893-b340-43eb-a9ea-b2636090cd63 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:07.498567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a0893-b340-43eb-a9ea-b2636090cd63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:07.499931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a0893-b340-43eb-a9ea-b2636090cd63 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:07.509043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 509/1011] 10.222.0.44 () {56 vars in 1122 bytes} [Mon May 9 04:14:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:07.510621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47831f33-60ca-43a5-92bf-432bbec528ff None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:07.605545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47831f33-60ca-43a5-92bf-432bbec528ff None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:07.637246 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d3fe0-377f-4a6b-8f28-5d4e0f291a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:07.638117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d3fe0-377f-4a6b-8f28-5d4e0f291a14 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:07.638406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d3fe0-377f-4a6b-8f28-5d4e0f291a14 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:07.638682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d3fe0-377f-4a6b-8f28-5d4e0f291a14 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:07.639383 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d3fe0-377f-4a6b-8f28-5d4e0f291a14 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:07.639686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d3fe0-377f-4a6b-8f28-5d4e0f291a14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:07.641061 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d3fe0-377f-4a6b-8f28-5d4e0f291a14 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:07.650202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 503/1012] 10.222.0.44 () {56 vars in 1122 bytes} [Mon May 9 04:14:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:07.651659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a0893-b340-43eb-a9ea-b2636090cd63 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:07.748585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a0893-b340-43eb-a9ea-b2636090cd63 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:07.782093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c3bc8-2112-42f1-a843-5fd55d230259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:07.782758 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3bc8-2112-42f1-a843-5fd55d230259 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:07.783040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3bc8-2112-42f1-a843-5fd55d230259 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:07.783312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3bc8-2112-42f1-a843-5fd55d230259 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:07.784088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c3bc8-2112-42f1-a843-5fd55d230259 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:07.785448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c3bc8-2112-42f1-a843-5fd55d230259 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:07.800596 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 510/1013] 10.222.0.44 () {60 vars in 1144 bytes} [Mon May 9 04:14:07 2022] POST /identity/v3/endpoints => generated 325 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:14:08.453829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d98bdb-db2f-4daa-b623-7726928db90b None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:08.454405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d98bdb-db2f-4daa-b623-7726928db90b None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:08.454648 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d98bdb-db2f-4daa-b623-7726928db90b None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:08.456068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 504/1014] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:08.461291 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bc617-0380-4ba4-8727-8528af366aed None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:08.461585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bc617-0380-4ba4-8727-8528af366aed None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:08.461860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bc617-0380-4ba4-8727-8528af366aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:08.478560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-418bc617-0380-4ba4-8727-8528af366aed None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:08.479718 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-418bc617-0380-4ba4-8727-8528af366aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:08.555922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 511/1015] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:08 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:14:08.574188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d98bdb-db2f-4daa-b623-7726928db90b None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:08.676305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d98bdb-db2f-4daa-b623-7726928db90b None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:08.707951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c388c64-58b9-412d-b404-b5a610a1fcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:08.708697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c388c64-58b9-412d-b404-b5a610a1fcff None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:08.708980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c388c64-58b9-412d-b404-b5a610a1fcff None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:08.709253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c388c64-58b9-412d-b404-b5a610a1fcff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:08.719428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c388c64-58b9-412d-b404-b5a610a1fcff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-9c388c64-58b9-412d-b404-b5a610a1fcff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:14:08.727510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:08.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:08.734819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:08.734819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:08.734819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:08.734819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:08.734819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:14:08.734819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:14:08.734819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:14:08.734819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:14:08.734819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:14:08.734819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:14:08.734819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 04:14:08.734819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:08.734819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 505/1016] 10.222.0.44 () {56 vars in 1120 bytes} [Mon May 9 04:14:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:08.735530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bc617-0380-4ba4-8727-8528af366aed None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:08.811131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad28fdf-acbc-41c4-ab16-d01f54979284 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:08.811450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad28fdf-acbc-41c4-ab16-d01f54979284 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:08.811675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad28fdf-acbc-41c4-ab16-d01f54979284 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:08.813195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 506/1017] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:14:08.817994 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbb589-03e7-4813-a0d0-a7102709e647 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:08.818278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbb589-03e7-4813-a0d0-a7102709e647 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:08.818555 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbb589-03e7-4813-a0d0-a7102709e647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:08.832861 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bc617-0380-4ba4-8727-8528af366aed None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:08.835238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-bbbbb589-03e7-4813-a0d0-a7102709e647 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:08.836420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-bbbbb589-03e7-4813-a0d0-a7102709e647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:08.865978 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d4ae-2c67-4280-9ae8-8e1cd1674e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:08.866643 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d4ae-2c67-4280-9ae8-8e1cd1674e28 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:08.866920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d4ae-2c67-4280-9ae8-8e1cd1674e28 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:08.867191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d4ae-2c67-4280-9ae8-8e1cd1674e28 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:08.867940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0d4ae-2c67-4280-9ae8-8e1cd1674e28 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:08.868245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0d4ae-2c67-4280-9ae8-8e1cd1674e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:08.869594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0d4ae-2c67-4280-9ae8-8e1cd1674e28 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:08.878511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 512/1018] 10.222.0.44 () {56 vars in 1115 bytes} [Mon May 9 04:14:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:08.883695 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d4ae-2c67-4280-9ae8-8e1cd1674e28 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:08.905789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 507/1019] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:08 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:14:08.928272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbb589-03e7-4813-a0d0-a7102709e647 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:08.984182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d4ae-2c67-4280-9ae8-8e1cd1674e28 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:09.018786 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77386b2f-2b3a-4aca-9365-07fd1eaa6b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:09.019585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77386b2f-2b3a-4aca-9365-07fd1eaa6b63 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:09.019951 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77386b2f-2b3a-4aca-9365-07fd1eaa6b63 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:09.020365 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77386b2f-2b3a-4aca-9365-07fd1eaa6b63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:09.030690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77386b2f-2b3a-4aca-9365-07fd1eaa6b63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:09.032156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77386b2f-2b3a-4aca-9365-07fd1eaa6b63 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:09.041712 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbb589-03e7-4813-a0d0-a7102709e647 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-77386b2f-2b3a-4aca-9365-07fd1eaa6b63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:14:09.050816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:09.051707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:09.057857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:14:09.057857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:14:09.057857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:09.057857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:09.057857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 513/1020] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:09.057857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77386b2f-2b3a-4aca-9365-07fd1eaa6b63 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:09.076128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae0499-3501-4137-a3e5-73d1a13ff434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:09.076858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae0499-3501-4137-a3e5-73d1a13ff434 None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:09.077145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae0499-3501-4137-a3e5-73d1a13ff434 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:09.077421 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae0499-3501-4137-a3e5-73d1a13ff434 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:09.078154 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae0499-3501-4137-a3e5-73d1a13ff434 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:09.079771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae0499-3501-4137-a3e5-73d1a13ff434 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:09.094191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 508/1021] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:14:08 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:14:09.168382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77386b2f-2b3a-4aca-9365-07fd1eaa6b63 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:09.210070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78959b99-477c-4ee2-abba-910d0ff06fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:09.210752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78959b99-477c-4ee2-abba-910d0ff06fc0 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:09.211037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78959b99-477c-4ee2-abba-910d0ff06fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:09.211314 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78959b99-477c-4ee2-abba-910d0ff06fc0 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:09.212071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78959b99-477c-4ee2-abba-910d0ff06fc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:09.212382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78959b99-477c-4ee2-abba-910d0ff06fc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:09.213770 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78959b99-477c-4ee2-abba-910d0ff06fc0 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:09.231248 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 514/1022] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:09.233207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae0499-3501-4137-a3e5-73d1a13ff434 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:09.331572 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae0499-3501-4137-a3e5-73d1a13ff434 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:09.363064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f93b7-59e5-49f3-b954-a5c3111f4745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:09.363856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f93b7-59e5-49f3-b954-a5c3111f4745 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:09.364162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f93b7-59e5-49f3-b954-a5c3111f4745 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:09.364435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f93b7-59e5-49f3-b954-a5c3111f4745 None admin] PATH_INFO: `/v3/users/nova` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:09.371720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f93b7-59e5-49f3-b954-a5c3111f4745 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:09.373743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f93b7-59e5-49f3-b954-a5c3111f4745 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-835f93b7-59e5-49f3-b954-a5c3111f4745 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:14:09.387142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:09.388385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:09.389345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:09.389345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:09.389345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:14:09.389345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:14:09.389345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:14:09.389345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:14:09.389345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:14:09.389345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:14:09.389345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 04:14:09.389345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:09.390293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 509/1023] 10.222.0.44 () {56 vars in 1093 bytes} [Mon May 9 04:14:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:09.394341 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78959b99-477c-4ee2-abba-910d0ff06fc0 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:09.488046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78959b99-477c-4ee2-abba-910d0ff06fc0 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:09.521419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d03b16-e3dd-428a-835e-7cb461189c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:09.522086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d03b16-e3dd-428a-835e-7cb461189c6b None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:09.522368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d03b16-e3dd-428a-835e-7cb461189c6b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:09.522639 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d03b16-e3dd-428a-835e-7cb461189c6b None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:09.523477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d03b16-e3dd-428a-835e-7cb461189c6b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:09.523783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d03b16-e3dd-428a-835e-7cb461189c6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:09.525227 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d03b16-e3dd-428a-835e-7cb461189c6b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:09.547329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 515/1024] 10.222.0.44 () {56 vars in 1133 bytes} [Mon May 9 04:14:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:09.548946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f93b7-59e5-49f3-b954-a5c3111f4745 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:09.644270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f93b7-59e5-49f3-b954-a5c3111f4745 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:09.675724 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137bc60-703e-49fc-85c5-8c68394a74f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:09.676406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137bc60-703e-49fc-85c5-8c68394a74f2 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:09.676688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137bc60-703e-49fc-85c5-8c68394a74f2 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:09.676962 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137bc60-703e-49fc-85c5-8c68394a74f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:09.747572 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137bc60-703e-49fc-85c5-8c68394a74f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:09.747572 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137bc60-703e-49fc-85c5-8c68394a74f2 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:09.839466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 510/1025] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:14:09 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:14:09.845617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d03b16-e3dd-428a-835e-7cb461189c6b None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:09.945948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d03b16-e3dd-428a-835e-7cb461189c6b None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:09.978109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9f837-e6fd-4644-8116-545654fcc383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:09.978898 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9f837-e6fd-4644-8116-545654fcc383 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:09.979175 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9f837-e6fd-4644-8116-545654fcc383 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:09.979446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9f837-e6fd-4644-8116-545654fcc383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:09.989815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc9f837-e6fd-4644-8116-545654fcc383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:09.991237 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc9f837-e6fd-4644-8116-545654fcc383 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-7dc9f837-e6fd-4644-8116-545654fcc383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:14:10.007783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:10.008805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:10.009730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:14:10.009730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:14:10.009730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:10.009730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:10.013692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 516/1026] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:10.015095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137bc60-703e-49fc-85c5-8c68394a74f2 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:10.081549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1c711-8b43-4ec2-bc3d-2fe49dd60acc None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:10.081869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1c711-8b43-4ec2-bc3d-2fe49dd60acc None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:10.082117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1c711-8b43-4ec2-bc3d-2fe49dd60acc None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:10.083642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 517/1027] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:10.088851 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85213d46-17ba-4062-a666-dad2b50de3b9 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:10.089280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85213d46-17ba-4062-a666-dad2b50de3b9 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:10.089468 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85213d46-17ba-4062-a666-dad2b50de3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:10.105963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-85213d46-17ba-4062-a666-dad2b50de3b9 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:10.107355 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-85213d46-17ba-4062-a666-dad2b50de3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:10.109052 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137bc60-703e-49fc-85c5-8c68394a74f2 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:10.141745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe69ba8-3bf2-4d7c-9410-80f43a8d7a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:10.142404 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe69ba8-3bf2-4d7c-9410-80f43a8d7a55 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:10.142686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe69ba8-3bf2-4d7c-9410-80f43a8d7a55 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:10.142959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe69ba8-3bf2-4d7c-9410-80f43a8d7a55 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:10.143675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe69ba8-3bf2-4d7c-9410-80f43a8d7a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:10.144007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe69ba8-3bf2-4d7c-9410-80f43a8d7a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:10.145443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe69ba8-3bf2-4d7c-9410-80f43a8d7a55 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:10.161778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 511/1028] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:10.162779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe69ba8-3bf2-4d7c-9410-80f43a8d7a55 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:10.179953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 518/1029] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:14:10.195115 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85213d46-17ba-4062-a666-dad2b50de3b9 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:10.259309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe69ba8-3bf2-4d7c-9410-80f43a8d7a55 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:10.292947 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89454d2f-4ce3-4a69-b7e4-d938e1a7e153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:10.293792 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89454d2f-4ce3-4a69-b7e4-d938e1a7e153 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:10.294083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89454d2f-4ce3-4a69-b7e4-d938e1a7e153 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:10.294262 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89454d2f-4ce3-4a69-b7e4-d938e1a7e153 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:10.296393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85213d46-17ba-4062-a666-dad2b50de3b9 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:10.307523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89454d2f-4ce3-4a69-b7e4-d938e1a7e153 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:10.309001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89454d2f-4ce3-4a69-b7e4-d938e1a7e153 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-89454d2f-4ce3-4a69-b7e4-d938e1a7e153 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:14:10.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:10.324363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:10.326450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:14:10.326450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:14:10.326450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:14:10.326450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:14:10.326450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:14:10.326450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:14:10.326450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:14:10.326450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:10.326450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 512/1030] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:14:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:10.331051 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89454d2f-4ce3-4a69-b7e4-d938e1a7e153 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:10.333678 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770e8a8-3b95-4173-a327-8f3937bdaa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:10.334479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770e8a8-3b95-4173-a327-8f3937bdaa48 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:10.334760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770e8a8-3b95-4173-a327-8f3937bdaa48 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:10.335019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770e8a8-3b95-4173-a327-8f3937bdaa48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:10.336209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2770e8a8-3b95-4173-a327-8f3937bdaa48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:10.337628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2770e8a8-3b95-4173-a327-8f3937bdaa48 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:10.339761 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 519/1031] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:10.345369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770e8a8-3b95-4173-a327-8f3937bdaa48 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:10.426517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89454d2f-4ce3-4a69-b7e4-d938e1a7e153 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:10.446839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770e8a8-3b95-4173-a327-8f3937bdaa48 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:10.459887 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c29a61-5b7f-49c1-b953-4b34e164831d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:10.460570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c29a61-5b7f-49c1-b953-4b34e164831d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:10.460854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c29a61-5b7f-49c1-b953-4b34e164831d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:10.461133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c29a61-5b7f-49c1-b953-4b34e164831d None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:10.461912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c29a61-5b7f-49c1-b953-4b34e164831d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:10.462225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c29a61-5b7f-49c1-b953-4b34e164831d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:10.463640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c29a61-5b7f-49c1-b953-4b34e164831d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:10.476123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 513/1032] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:14:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:10.481683 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2d730-cb43-47e7-84e8-7a87b61dea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:10.482382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2d730-cb43-47e7-84e8-7a87b61dea72 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:10.482668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2d730-cb43-47e7-84e8-7a87b61dea72 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:10.482868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c29a61-5b7f-49c1-b953-4b34e164831d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:10.482968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2d730-cb43-47e7-84e8-7a87b61dea72 None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:10.483638 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2d730-cb43-47e7-84e8-7a87b61dea72 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:10.485097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2d730-cb43-47e7-84e8-7a87b61dea72 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:10.510012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 520/1033] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:14:10 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:14:10.598935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c29a61-5b7f-49c1-b953-4b34e164831d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:10.633587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933116e-9ac6-475f-bd2e-4c036e799424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:10.634605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933116e-9ac6-475f-bd2e-4c036e799424 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:10.635069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933116e-9ac6-475f-bd2e-4c036e799424 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:10.635520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933116e-9ac6-475f-bd2e-4c036e799424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:10.636569 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933116e-9ac6-475f-bd2e-4c036e799424 None admin] RBAC: Adding query filter params (role.id=7de5200fcd0943d9834b6e12861d5941, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=faf6107735424ff7901614953e7df403) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:10.637066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933116e-9ac6-475f-bd2e-4c036e799424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:10.639057 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933116e-9ac6-475f-bd2e-4c036e799424 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:10.645911 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 514/1034] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:14:10 2022] GET /identity/v3/role_assignments?user.id=faf6107735424ff7901614953e7df403&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=7de5200fcd0943d9834b6e12861d5941 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:11.367373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928e2dd-66bd-4c14-b876-e0b4c55364bd None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:11.367941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928e2dd-66bd-4c14-b876-e0b4c55364bd None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:11.368401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928e2dd-66bd-4c14-b876-e0b4c55364bd None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:11.369917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 521/1035] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:11.375507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b4190-49d3-49ef-a221-0a461f6ebc8c None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:11.376135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b4190-49d3-49ef-a221-0a461f6ebc8c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:11.376688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b4190-49d3-49ef-a221-0a461f6ebc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:11.395695 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-914b4190-49d3-49ef-a221-0a461f6ebc8c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:11.397682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-914b4190-49d3-49ef-a221-0a461f6ebc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:11.470729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 515/1036] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:14:11.493415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928e2dd-66bd-4c14-b876-e0b4c55364bd None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:11.589033 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928e2dd-66bd-4c14-b876-e0b4c55364bd None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:11.620225 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057970f8-49db-419e-9d3b-6af067f77814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:11.620945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057970f8-49db-419e-9d3b-6af067f77814 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:11.621227 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057970f8-49db-419e-9d3b-6af067f77814 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:11.621501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057970f8-49db-419e-9d3b-6af067f77814 None admin] PATH_INFO: `/v3/services/image` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:11.622176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057970f8-49db-419e-9d3b-6af067f77814 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:11.623535 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057970f8-49db-419e-9d3b-6af067f77814 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-057970f8-49db-419e-9d3b-6af067f77814 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:11.634663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:11.635637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 04:14:11.636539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:11.640208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 522/1037] 10.222.0.44 () {56 vars in 1105 bytes} [Mon May 9 04:14:11 2022] GET /identity/v3/services/image => generated 86 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:11.642282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b4190-49d3-49ef-a221-0a461f6ebc8c None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:11.734907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b4190-49d3-49ef-a221-0a461f6ebc8c None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:11.783546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3c89d-73f9-4d13-9682-d792c24d12ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:11.787514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3c89d-73f9-4d13-9682-d792c24d12ea None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:11.787514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3c89d-73f9-4d13-9682-d792c24d12ea None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:11.787514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3c89d-73f9-4d13-9682-d792c24d12ea None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:11.787514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3c89d-73f9-4d13-9682-d792c24d12ea None admin] RBAC: Adding query filter params (name=image) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:11.787514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3c89d-73f9-4d13-9682-d792c24d12ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:11.787514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3c89d-73f9-4d13-9682-d792c24d12ea None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:11.797433 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 516/1038] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:11.798774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057970f8-49db-419e-9d3b-6af067f77814 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:11.901098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057970f8-49db-419e-9d3b-6af067f77814 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:11.932924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec141d-5d8a-4fa8-9cf6-e0f02f3b0bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:11.933920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec141d-5d8a-4fa8-9cf6-e0f02f3b0bef None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:11.935051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec141d-5d8a-4fa8-9cf6-e0f02f3b0bef None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:11.935585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec141d-5d8a-4fa8-9cf6-e0f02f3b0bef None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:11.936520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec141d-5d8a-4fa8-9cf6-e0f02f3b0bef None admin] RBAC: Adding query filter params (type=image) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:11.937040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec141d-5d8a-4fa8-9cf6-e0f02f3b0bef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:11.938589 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec141d-5d8a-4fa8-9cf6-e0f02f3b0bef None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:11.943511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 523/1039] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:11.949479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3c89d-73f9-4d13-9682-d792c24d12ea None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:12.042858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3c89d-73f9-4d13-9682-d792c24d12ea None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:12.075082 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c7ddd-3e98-4967-b8b4-05d01a1fc969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:12.075787 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c7ddd-3e98-4967-b8b4-05d01a1fc969 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:12.076104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c7ddd-3e98-4967-b8b4-05d01a1fc969 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:12.076380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c7ddd-3e98-4967-b8b4-05d01a1fc969 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:12.077159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c7ddd-3e98-4967-b8b4-05d01a1fc969 None admin] RBAC: Adding query filter params (interface=public, service_id=b0002af7868243faa9c631696d1f23aa, region_id=RegionOne) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:12.077466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c7ddd-3e98-4967-b8b4-05d01a1fc969 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:12.078812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c7ddd-3e98-4967-b8b4-05d01a1fc969 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:12.088426 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 517/1040] 10.222.0.44 () {56 vars in 1251 bytes} [Mon May 9 04:14:11 2022] GET /identity/v3/endpoints?service_id=b0002af7868243faa9c631696d1f23aa&interface=public®ion_id=RegionOne => generated 194 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:12.089980 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec141d-5d8a-4fa8-9cf6-e0f02f3b0bef None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:12.184746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec141d-5d8a-4fa8-9cf6-e0f02f3b0bef None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:12.216710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef341253-c63f-4e6f-b527-b6b08b93aad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:12.217374 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef341253-c63f-4e6f-b527-b6b08b93aad2 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:12.217656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef341253-c63f-4e6f-b527-b6b08b93aad2 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:12.217929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef341253-c63f-4e6f-b527-b6b08b93aad2 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:12.218604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef341253-c63f-4e6f-b527-b6b08b93aad2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:12.219983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef341253-c63f-4e6f-b527-b6b08b93aad2 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:12.231785 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 524/1041] 10.222.0.44 () {56 vars in 1087 bytes} [Mon May 9 04:14:12 2022] GET /identity/v3/services => generated 1957 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:14:12.858521 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb711cc-cdd5-4be4-93b1-a5d31934c133 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:12.859039 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb711cc-cdd5-4be4-93b1-a5d31934c133 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:12.859115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb711cc-cdd5-4be4-93b1-a5d31934c133 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:12.860587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 518/1042] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:12.865641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc4c77-34ab-4ec3-8887-f2e6bdc30f78 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:12.866025 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc4c77-34ab-4ec3-8887-f2e6bdc30f78 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:12.866191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc4c77-34ab-4ec3-8887-f2e6bdc30f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:12.882904 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c4dc4c77-34ab-4ec3-8887-f2e6bdc30f78 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:12.884086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c4dc4c77-34ab-4ec3-8887-f2e6bdc30f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:12.908871 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d674e3c-f3ef-4a76-9a25-5e875f8ce77d None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:12.909169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d674e3c-f3ef-4a76-9a25-5e875f8ce77d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:12.909444 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d674e3c-f3ef-4a76-9a25-5e875f8ce77d None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:12.910916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 519/1043] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:14:12.915577 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5650b89-4d8d-4b1e-9250-bbcbce56075a None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:12.915853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5650b89-4d8d-4b1e-9250-bbcbce56075a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:12.916167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5650b89-4d8d-4b1e-9250-bbcbce56075a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:12.932228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-a5650b89-4d8d-4b1e-9250-bbcbce56075a None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:12.933403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a5650b89-4d8d-4b1e-9250-bbcbce56075a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:12.952046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 525/1044] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:12.972373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc4c77-34ab-4ec3-8887-f2e6bdc30f78 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:12.998322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 520/1045] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:14:13.021546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5650b89-4d8d-4b1e-9250-bbcbce56075a None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:13.066854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc4c77-34ab-4ec3-8887-f2e6bdc30f78 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:13.107161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba2167-adde-4b94-970c-94f499543b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:13.107985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba2167-adde-4b94-970c-94f499543b61 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:13.108263 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba2167-adde-4b94-970c-94f499543b61 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:13.108529 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba2167-adde-4b94-970c-94f499543b61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:13.109698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba2167-adde-4b94-970c-94f499543b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:13.111182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba2167-adde-4b94-970c-94f499543b61 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:13.112926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 526/1046] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:13.118173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba2167-adde-4b94-970c-94f499543b61 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:13.121214 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5650b89-4d8d-4b1e-9250-bbcbce56075a None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:13.155380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e585b-ef1f-4fb2-9b96-0bc5c4d4b01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:13.156553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e585b-ef1f-4fb2-9b96-0bc5c4d4b01a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:13.156841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e585b-ef1f-4fb2-9b96-0bc5c4d4b01a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:13.157112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e585b-ef1f-4fb2-9b96-0bc5c4d4b01a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:13.167211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e585b-ef1f-4fb2-9b96-0bc5c4d4b01a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-806e585b-ef1f-4fb2-9b96-0bc5c4d4b01a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:14:13.176035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:13.176977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:13.177805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:13.177805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:13.177805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:13.177805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:13.177805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:14:13.177805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:14:13.177805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:14:13.177805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:14:13.177805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:14:13.177805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:14:13.177805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 04:14:13.177805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:13.180209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 521/1047] 10.222.0.44 () {56 vars in 1120 bytes} [Mon May 9 04:14:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:13.182933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e585b-ef1f-4fb2-9b96-0bc5c4d4b01a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:13.214146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba2167-adde-4b94-970c-94f499543b61 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:13.246299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7020c-17aa-47e0-a8b6-1499b2b72b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:13.246987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7020c-17aa-47e0-a8b6-1499b2b72b21 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:13.247269 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7020c-17aa-47e0-a8b6-1499b2b72b21 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:13.247543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7020c-17aa-47e0-a8b6-1499b2b72b21 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:13.248275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7020c-17aa-47e0-a8b6-1499b2b72b21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:13.249715 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7020c-17aa-47e0-a8b6-1499b2b72b21 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:13.269803 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 527/1048] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:14:13 2022] POST /identity/v3/users => generated 285 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:14:13.283367 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e585b-ef1f-4fb2-9b96-0bc5c4d4b01a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:13.320054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328eb0-ce68-4605-839c-0d22c6996a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:13.320733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328eb0-ce68-4605-839c-0d22c6996a62 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:13.321018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328eb0-ce68-4605-839c-0d22c6996a62 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:13.321291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328eb0-ce68-4605-839c-0d22c6996a62 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:13.322045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e328eb0-ce68-4605-839c-0d22c6996a62 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:13.322352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e328eb0-ce68-4605-839c-0d22c6996a62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:13.323738 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e328eb0-ce68-4605-839c-0d22c6996a62 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:13.338776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 522/1049] 10.222.0.44 () {56 vars in 1115 bytes} [Mon May 9 04:14:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:13.339925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7020c-17aa-47e0-a8b6-1499b2b72b21 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:13.446263 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7020c-17aa-47e0-a8b6-1499b2b72b21 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:13.483040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bae597-bbd1-4c13-b640-3aaae030dbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:13.483844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bae597-bbd1-4c13-b640-3aaae030dbce None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:13.484148 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bae597-bbd1-4c13-b640-3aaae030dbce None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:13.484422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bae597-bbd1-4c13-b640-3aaae030dbce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:13.494895 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bae597-bbd1-4c13-b640-3aaae030dbce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:13.496391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bae597-bbd1-4c13-b640-3aaae030dbce None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-91bae597-bbd1-4c13-b640-3aaae030dbce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:14:13.514967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:13.516132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:13.517224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:14:13.517224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:14:13.517224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:13.517224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:13.519059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 528/1050] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:13.524862 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328eb0-ce68-4605-839c-0d22c6996a62 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:13.621675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328eb0-ce68-4605-839c-0d22c6996a62 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:13.653446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e28917-57f1-43f7-b9c9-997485e04f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:13.654356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e28917-57f1-43f7-b9c9-997485e04f9b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:13.654807 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e28917-57f1-43f7-b9c9-997485e04f9b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:13.655221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e28917-57f1-43f7-b9c9-997485e04f9b None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:13.656100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e28917-57f1-43f7-b9c9-997485e04f9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:13.656561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e28917-57f1-43f7-b9c9-997485e04f9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:13.658048 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e28917-57f1-43f7-b9c9-997485e04f9b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:13.670670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 523/1051] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:13.676330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bae597-bbd1-4c13-b640-3aaae030dbce None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:13.780187 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bae597-bbd1-4c13-b640-3aaae030dbce None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:13.811517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec24837-81e4-4761-99ba-a9a136ac3eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:13.812338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec24837-81e4-4761-99ba-a9a136ac3eef None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:13.812618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec24837-81e4-4761-99ba-a9a136ac3eef None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:13.812887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec24837-81e4-4761-99ba-a9a136ac3eef None admin] PATH_INFO: `/v3/users/nova` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:13.820142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec24837-81e4-4761-99ba-a9a136ac3eef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:13.821567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec24837-81e4-4761-99ba-a9a136ac3eef None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-bec24837-81e4-4761-99ba-a9a136ac3eef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:14:13.835855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:13.836836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:13.837822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:13.837822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:13.837822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:14:13.837822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:14:13.837822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:14:13.837822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:14:13.837822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:14:13.837822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:14:13.837822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 04:14:13.837822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:13.839977 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 529/1052] 10.222.0.44 () {56 vars in 1093 bytes} [Mon May 9 04:14:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:13.842942 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e28917-57f1-43f7-b9c9-997485e04f9b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:13.942506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e28917-57f1-43f7-b9c9-997485e04f9b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:13.974571 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0927064-2580-47e0-8fd1-cdb136eae91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:13.975312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0927064-2580-47e0-8fd1-cdb136eae91a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:13.975597 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0927064-2580-47e0-8fd1-cdb136eae91a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:13.975870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0927064-2580-47e0-8fd1-cdb136eae91a None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:13.976715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0927064-2580-47e0-8fd1-cdb136eae91a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:13.977023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0927064-2580-47e0-8fd1-cdb136eae91a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:13.978427 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0927064-2580-47e0-8fd1-cdb136eae91a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:13.996455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 524/1053] 10.222.0.44 () {56 vars in 1133 bytes} [Mon May 9 04:14:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:14.002294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec24837-81e4-4761-99ba-a9a136ac3eef None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:14.101612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec24837-81e4-4761-99ba-a9a136ac3eef None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:14.134310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3351f8e-570e-4757-899a-447d85c3b529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:14.135105 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3351f8e-570e-4757-899a-447d85c3b529 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:14.135397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3351f8e-570e-4757-899a-447d85c3b529 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:14.135660 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3351f8e-570e-4757-899a-447d85c3b529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:14.146969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3351f8e-570e-4757-899a-447d85c3b529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:14.148441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3351f8e-570e-4757-899a-447d85c3b529 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-e3351f8e-570e-4757-899a-447d85c3b529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:14:14.165945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:14.166836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:14.167729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:14:14.167729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:14:14.167729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:14.167729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:14.168638 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 530/1054] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:14.173013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0927064-2580-47e0-8fd1-cdb136eae91a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:14.266161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0927064-2580-47e0-8fd1-cdb136eae91a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:14.297793 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f366ac-1121-4df1-9583-e808b8bf082d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:14.298481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f366ac-1121-4df1-9583-e808b8bf082d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:14.298759 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f366ac-1121-4df1-9583-e808b8bf082d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:14.299034 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f366ac-1121-4df1-9583-e808b8bf082d None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:14.299767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f366ac-1121-4df1-9583-e808b8bf082d None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:14.300101 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f366ac-1121-4df1-9583-e808b8bf082d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:14.301490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f366ac-1121-4df1-9583-e808b8bf082d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:14.318963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 525/1055] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:14.320266 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3351f8e-570e-4757-899a-447d85c3b529 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:14.414870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3351f8e-570e-4757-899a-447d85c3b529 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:14.446119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf3466-71af-48a6-a6ac-54757fb5411c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:14.446918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf3466-71af-48a6-a6ac-54757fb5411c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:14.447194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf3466-71af-48a6-a6ac-54757fb5411c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:14.447465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf3466-71af-48a6-a6ac-54757fb5411c None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:14.454978 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf3466-71af-48a6-a6ac-54757fb5411c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:14.456481 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf3466-71af-48a6-a6ac-54757fb5411c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-87cf3466-71af-48a6-a6ac-54757fb5411c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:14:14.471654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:14.472690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:14.473595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:14:14.473595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:14:14.473595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:14:14.473595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:14:14.473595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:14:14.473595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:14:14.473595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:14:14.473595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:14.473595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 531/1056] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:14:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:14.479959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f366ac-1121-4df1-9583-e808b8bf082d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:14.489062 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec0c8e-01a6-4981-ac3d-4c0850605b92 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:14.489356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec0c8e-01a6-4981-ac3d-4c0850605b92 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:14.489641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec0c8e-01a6-4981-ac3d-4c0850605b92 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:14.491018 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 532/1057] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:14:14.497211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4250407-f379-465a-be57-56f97eed1ca5 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:14.497658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4250407-f379-465a-be57-56f97eed1ca5 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:14.497942 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4250407-f379-465a-be57-56f97eed1ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:14.514848 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-f4250407-f379-465a-be57-56f97eed1ca5 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:14.516051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f4250407-f379-465a-be57-56f97eed1ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:14.578029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f366ac-1121-4df1-9583-e808b8bf082d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:14.581283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 533/1058] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:14:14.603709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4250407-f379-465a-be57-56f97eed1ca5 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:14.610295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e86aa-8063-4417-a042-dae042048450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:14.611221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e86aa-8063-4417-a042-dae042048450 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:14.611726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e86aa-8063-4417-a042-dae042048450 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:14.612058 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e86aa-8063-4417-a042-dae042048450 None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:14.612852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e86aa-8063-4417-a042-dae042048450 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:14.613168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e86aa-8063-4417-a042-dae042048450 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:14.614586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e86aa-8063-4417-a042-dae042048450 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:14.627153 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 526/1059] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:14:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:14.632884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e86aa-8063-4417-a042-dae042048450 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:14.699403 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4250407-f379-465a-be57-56f97eed1ca5 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:14.729000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e86aa-8063-4417-a042-dae042048450 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:14.733619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152dfb6-29bb-4cff-9654-16b863d7dcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:14.734402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152dfb6-29bb-4cff-9654-16b863d7dcc2 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:14.734679 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152dfb6-29bb-4cff-9654-16b863d7dcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:14.734953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152dfb6-29bb-4cff-9654-16b863d7dcc2 None admin] PATH_INFO: `/v3/services/image` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:14.735593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152dfb6-29bb-4cff-9654-16b863d7dcc2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:14.737065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152dfb6-29bb-4cff-9654-16b863d7dcc2 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-0152dfb6-29bb-4cff-9654-16b863d7dcc2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:14.748896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:14.749914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:14.759040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 534/1060] 10.222.0.44 () {56 vars in 1105 bytes} [Mon May 9 04:14:14 2022] GET /identity/v3/services/image => generated 86 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:14.761710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152dfb6-29bb-4cff-9654-16b863d7dcc2 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:14.767675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daf667-bfca-4e0f-8a8d-76f2753d53eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:14.768775 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daf667-bfca-4e0f-8a8d-76f2753d53eb None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:14.769162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daf667-bfca-4e0f-8a8d-76f2753d53eb None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:14.769537 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47daf667-bfca-4e0f-8a8d-76f2753d53eb None admin] PATH_INFO: `/v3/projects/3322b00723974bd99e837d447bac22d1/users/faf6107735424ff7901614953e7df403/roles/7de5200fcd0943d9834b6e12861d5941` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:14.772373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47daf667-bfca-4e0f-8a8d-76f2753d53eb None admin] RBAC: Authorizing `identity:create_grant(project_id=3322b00723974bd99e837d447bac22d1, user_id=faf6107735424ff7901614953e7df403, role_id=7de5200fcd0943d9834b6e12861d5941)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:14.773796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47daf667-bfca-4e0f-8a8d-76f2753d53eb None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:14.789815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 527/1061] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:14:14 2022] PUT /identity/v3/projects/3322b00723974bd99e837d447bac22d1/users/faf6107735424ff7901614953e7df403/roles/7de5200fcd0943d9834b6e12861d5941 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:14:14.876532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152dfb6-29bb-4cff-9654-16b863d7dcc2 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:14.910540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd842a-f8bc-415d-8b13-67d5cff3ca33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:14.911216 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd842a-f8bc-415d-8b13-67d5cff3ca33 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:14.911493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd842a-f8bc-415d-8b13-67d5cff3ca33 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:14.911764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd842a-f8bc-415d-8b13-67d5cff3ca33 None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:14.912510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd842a-f8bc-415d-8b13-67d5cff3ca33 None admin] RBAC: Adding query filter params (name=image) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:14.912810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd842a-f8bc-415d-8b13-67d5cff3ca33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:14.914215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd842a-f8bc-415d-8b13-67d5cff3ca33 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:14.924961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 535/1062] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:14.926711 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daf667-bfca-4e0f-8a8d-76f2753d53eb None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:15.028783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47daf667-bfca-4e0f-8a8d-76f2753d53eb None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:15.064955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31130be2-44f0-4a5c-ac33-e6a129a35016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:15.065628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31130be2-44f0-4a5c-ac33-e6a129a35016 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:15.065907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31130be2-44f0-4a5c-ac33-e6a129a35016 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:15.066181 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31130be2-44f0-4a5c-ac33-e6a129a35016 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:15.066887 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31130be2-44f0-4a5c-ac33-e6a129a35016 None admin] RBAC: Adding query filter params (type=image) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:15.067192 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31130be2-44f0-4a5c-ac33-e6a129a35016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:15.068569 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31130be2-44f0-4a5c-ac33-e6a129a35016 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:15.073439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 528/1063] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:15.080162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd842a-f8bc-415d-8b13-67d5cff3ca33 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:15.179708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd842a-f8bc-415d-8b13-67d5cff3ca33 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:15.213544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c18dc-c876-469a-9f2e-e65d1ce9205d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:15.214204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c18dc-c876-469a-9f2e-e65d1ce9205d None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:15.214491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c18dc-c876-469a-9f2e-e65d1ce9205d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:15.214767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c18dc-c876-469a-9f2e-e65d1ce9205d None admin] PATH_INFO: `/v3/endpoints` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:15.215719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c18dc-c876-469a-9f2e-e65d1ce9205d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:15.217651 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c18dc-c876-469a-9f2e-e65d1ce9205d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:15.243918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 536/1064] 10.222.0.44 () {60 vars in 1144 bytes} [Mon May 9 04:14:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:14:15.601452 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91887447-53a1-4c41-907e-0f5e5173522b None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:15.601803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91887447-53a1-4c41-907e-0f5e5173522b None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:15.602023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91887447-53a1-4c41-907e-0f5e5173522b None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:15.603518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 529/1065] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:15.608735 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f178a1-3a3b-44e1-a2c7-2a104b2ff629 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:15.609029 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f178a1-3a3b-44e1-a2c7-2a104b2ff629 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:15.609308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f178a1-3a3b-44e1-a2c7-2a104b2ff629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:15.625514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-97f178a1-3a3b-44e1-a2c7-2a104b2ff629 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:15.626684 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-97f178a1-3a3b-44e1-a2c7-2a104b2ff629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:15.696265 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 537/1066] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:15.720095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91887447-53a1-4c41-907e-0f5e5173522b None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:15.823356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91887447-53a1-4c41-907e-0f5e5173522b None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:15.855699 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e69646-f80a-4907-88eb-0229e29defb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:15.856680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e69646-f80a-4907-88eb-0229e29defb1 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:15.856965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e69646-f80a-4907-88eb-0229e29defb1 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:15.857241 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e69646-f80a-4907-88eb-0229e29defb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:15.866868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e69646-f80a-4907-88eb-0229e29defb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-18e69646-f80a-4907-88eb-0229e29defb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:14:15.874872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:15.875794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:15.876673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:15.876673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:15.876673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:15.876673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:15.876673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:14:15.876673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:14:15.876673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:14:15.876673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:14:15.876673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:14:15.876673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:14:15.876673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:14:15.876673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:15.877949 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 530/1067] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:14:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:15.881978 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f178a1-3a3b-44e1-a2c7-2a104b2ff629 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:15.976423 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f178a1-3a3b-44e1-a2c7-2a104b2ff629 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:16.009040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9410962-2eff-4433-a604-2d70136c2065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:16.009716 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9410962-2eff-4433-a604-2d70136c2065 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:16.009995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9410962-2eff-4433-a604-2d70136c2065 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:16.010284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9410962-2eff-4433-a604-2d70136c2065 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:16.011019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9410962-2eff-4433-a604-2d70136c2065 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:16.011321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9410962-2eff-4433-a604-2d70136c2065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:16.013273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9410962-2eff-4433-a604-2d70136c2065 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:16.021436 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 538/1068] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:14:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:16.026902 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e69646-f80a-4907-88eb-0229e29defb1 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:16.125518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e69646-f80a-4907-88eb-0229e29defb1 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:16.157333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f38268-9526-49f1-af71-442c63b4f86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:16.158249 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f38268-9526-49f1-af71-442c63b4f86d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:16.158570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f38268-9526-49f1-af71-442c63b4f86d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:16.158848 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f38268-9526-49f1-af71-442c63b4f86d None admin] PATH_INFO: `/v3/users/d0c6e49d332744289bde848c528bf66c` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:16.172911 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f38268-9526-49f1-af71-442c63b4f86d None admin] RBAC: Authorizing `identity:get_user(user_id=d0c6e49d332744289bde848c528bf66c)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:16.174344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f38268-9526-49f1-af71-442c63b4f86d None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:16.180657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 531/1069] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:14:16 2022] GET /identity/v3/users/d0c6e49d332744289bde848c528bf66c => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:16.181696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9410962-2eff-4433-a604-2d70136c2065 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:16.277164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9410962-2eff-4433-a604-2d70136c2065 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:16.308438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3fed3-3fd2-4f0a-ae1f-4ab7bb9d2a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:16.309145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3fed3-3fd2-4f0a-ae1f-4ab7bb9d2a91 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:16.309642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3fed3-3fd2-4f0a-ae1f-4ab7bb9d2a91 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:16.310001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3fed3-3fd2-4f0a-ae1f-4ab7bb9d2a91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:16.381954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3fed3-3fd2-4f0a-ae1f-4ab7bb9d2a91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:16.383353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3fed3-3fd2-4f0a-ae1f-4ab7bb9d2a91 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:16.483924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 539/1070] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:14:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 04:14:16.486185 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f38268-9526-49f1-af71-442c63b4f86d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:16.581036 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f38268-9526-49f1-af71-442c63b4f86d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:16.612807 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68970875-61d3-4a7d-b484-f53e6382706c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:16.613674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68970875-61d3-4a7d-b484-f53e6382706c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:16.613955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68970875-61d3-4a7d-b484-f53e6382706c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:16.614230 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68970875-61d3-4a7d-b484-f53e6382706c None admin] PATH_INFO: `/v3/projects/eb5f521580e74518821f8f0072d10858` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:16.615405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68970875-61d3-4a7d-b484-f53e6382706c None admin] RBAC: Authorizing `identity:get_project(project_id=eb5f521580e74518821f8f0072d10858)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:16.616849 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68970875-61d3-4a7d-b484-f53e6382706c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:16.618733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 532/1071] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:14:16 2022] GET /identity/v3/projects/eb5f521580e74518821f8f0072d10858 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:16.625156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3fed3-3fd2-4f0a-ae1f-4ab7bb9d2a91 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:16.718067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3fed3-3fd2-4f0a-ae1f-4ab7bb9d2a91 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:16.750506 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778612d-ad87-4d44-9376-dadbc22b6be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:16.751156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778612d-ad87-4d44-9376-dadbc22b6be8 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:16.751436 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778612d-ad87-4d44-9376-dadbc22b6be8 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:16.751709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778612d-ad87-4d44-9376-dadbc22b6be8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:16.752697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778612d-ad87-4d44-9376-dadbc22b6be8 None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=eb5f521580e74518821f8f0072d10858, user.id=d0c6e49d332744289bde848c528bf66c) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:16.753006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778612d-ad87-4d44-9376-dadbc22b6be8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:16.756872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778612d-ad87-4d44-9376-dadbc22b6be8 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:16.760945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 540/1072] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:14:16 2022] GET /identity/v3/role_assignments?user.id=d0c6e49d332744289bde848c528bf66c&scope.project.id=eb5f521580e74518821f8f0072d10858&role.id=e59113c0366049e889df3f55eed90c72 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:17.064431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7f112-46be-4897-b931-50484c014230 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:17.064725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7f112-46be-4897-b931-50484c014230 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:17.064910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7f112-46be-4897-b931-50484c014230 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:17.066256 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 533/1073] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:17.071526 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194eab7-0889-4ea9-bf1e-286ea50e2082 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:17.072141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194eab7-0889-4ea9-bf1e-286ea50e2082 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:17.072511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194eab7-0889-4ea9-bf1e-286ea50e2082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:17.089894 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-1194eab7-0889-4ea9-bf1e-286ea50e2082 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:17.091839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1194eab7-0889-4ea9-bf1e-286ea50e2082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:17.155441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 541/1074] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:14:17.178360 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7f112-46be-4897-b931-50484c014230 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:17.276425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7f112-46be-4897-b931-50484c014230 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:17.308176 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512bb7b-2986-4e8b-8ac0-96b036ce8a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:17.309567 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512bb7b-2986-4e8b-8ac0-96b036ce8a6c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:17.309892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512bb7b-2986-4e8b-8ac0-96b036ce8a6c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:17.310123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512bb7b-2986-4e8b-8ac0-96b036ce8a6c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:17.321966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512bb7b-2986-4e8b-8ac0-96b036ce8a6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-9512bb7b-2986-4e8b-8ac0-96b036ce8a6c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:14:17.331072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:17.332406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:17.338339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:17.338339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:17.338339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:17.338339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:17.338339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:14:17.338339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:14:17.338339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:14:17.338339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:14:17.338339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:14:17.338339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:14:17.338339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 04:14:17.338339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:17.338339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 534/1075] 10.222.0.44 () {56 vars in 1120 bytes} [Mon May 9 04:14:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:17.340937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194eab7-0889-4ea9-bf1e-286ea50e2082 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:17.433032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194eab7-0889-4ea9-bf1e-286ea50e2082 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:17.466066 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba045dd-068d-482a-aa24-7003fc73cc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:17.466642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba045dd-068d-482a-aa24-7003fc73cc79 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:17.466924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba045dd-068d-482a-aa24-7003fc73cc79 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:17.467199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba045dd-068d-482a-aa24-7003fc73cc79 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:17.467967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba045dd-068d-482a-aa24-7003fc73cc79 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:17.468272 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba045dd-068d-482a-aa24-7003fc73cc79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:17.469626 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba045dd-068d-482a-aa24-7003fc73cc79 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:17.477659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 542/1076] 10.222.0.44 () {56 vars in 1115 bytes} [Mon May 9 04:14:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:17.483289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512bb7b-2986-4e8b-8ac0-96b036ce8a6c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:17.508915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d472257-7cdd-4e7d-a5e3-dd3ff5f0fcc0 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:17.509199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d472257-7cdd-4e7d-a5e3-dd3ff5f0fcc0 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:17.509472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d472257-7cdd-4e7d-a5e3-dd3ff5f0fcc0 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:17.510822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 543/1077] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:14:17.517219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a37e3-03d9-4711-8423-72706c73080f None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:17.517512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a37e3-03d9-4711-8423-72706c73080f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:17.517796 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a37e3-03d9-4711-8423-72706c73080f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:17.534298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-521a37e3-03d9-4711-8423-72706c73080f None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:17.535489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-521a37e3-03d9-4711-8423-72706c73080f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:17.583127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512bb7b-2986-4e8b-8ac0-96b036ce8a6c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:17.601230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 544/1078] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:14:17.615220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b24d25-fdc1-4c82-85e5-35f1325e4403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:17.616127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b24d25-fdc1-4c82-85e5-35f1325e4403 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:17.616477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b24d25-fdc1-4c82-85e5-35f1325e4403 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:17.616822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b24d25-fdc1-4c82-85e5-35f1325e4403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:17.624549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a37e3-03d9-4711-8423-72706c73080f None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:17.627521 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b24d25-fdc1-4c82-85e5-35f1325e4403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:17.628990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b24d25-fdc1-4c82-85e5-35f1325e4403 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-d9b24d25-fdc1-4c82-85e5-35f1325e4403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:14:17.645270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:17.646275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:17.647189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:14:17.647189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:14:17.647189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:17.647189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:17.647189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 535/1079] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:17.651961 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b24d25-fdc1-4c82-85e5-35f1325e4403 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:17.722422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a37e3-03d9-4711-8423-72706c73080f None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:17.748672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b24d25-fdc1-4c82-85e5-35f1325e4403 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:17.758860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b4a90-da26-42d3-afb2-1db2455fa4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:17.759748 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b4a90-da26-42d3-afb2-1db2455fa4ae None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:17.760112 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b4a90-da26-42d3-afb2-1db2455fa4ae None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:17.760537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b4a90-da26-42d3-afb2-1db2455fa4ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:17.783454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c9332-b373-4fbf-9964-fccb40391d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:17.784380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9332-b373-4fbf-9964-fccb40391d81 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:17.784665 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9332-b373-4fbf-9964-fccb40391d81 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:17.785042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9332-b373-4fbf-9964-fccb40391d81 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:17.785888 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c9332-b373-4fbf-9964-fccb40391d81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:17.785996 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c9332-b373-4fbf-9964-fccb40391d81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:17.787315 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c9332-b373-4fbf-9964-fccb40391d81 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:17.800226 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 536/1080] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:17.805564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c9332-b373-4fbf-9964-fccb40391d81 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:17.827212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b4a90-da26-42d3-afb2-1db2455fa4ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:17.828699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b4a90-da26-42d3-afb2-1db2455fa4ae None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:17.902309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c9332-b373-4fbf-9964-fccb40391d81 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:17.925764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 545/1081] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:14:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 04:14:17.931296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b4a90-da26-42d3-afb2-1db2455fa4ae None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:17.938309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9233c-46bc-4856-a9c2-96f4c9744bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:17.939110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9233c-46bc-4856-a9c2-96f4c9744bfb None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:17.939392 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9233c-46bc-4856-a9c2-96f4c9744bfb None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:17.939668 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9233c-46bc-4856-a9c2-96f4c9744bfb None admin] PATH_INFO: `/v3/users/nova` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:17.946822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9233c-46bc-4856-a9c2-96f4c9744bfb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:17.948289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9233c-46bc-4856-a9c2-96f4c9744bfb None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-34e9233c-46bc-4856-a9c2-96f4c9744bfb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:14:17.966776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:17.967767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:17.969117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:17.969117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:17.969117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:14:17.969117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:14:17.969117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:14:17.969117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:14:17.969117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:14:17.969117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:14:17.969117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 04:14:17.969117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:17.972248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 537/1082] 10.222.0.44 () {56 vars in 1093 bytes} [Mon May 9 04:14:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:17.979366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9233c-46bc-4856-a9c2-96f4c9744bfb None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:18.030187 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b4a90-da26-42d3-afb2-1db2455fa4ae None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:18.065959 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b69c1-4400-42c3-afdc-c60cffe9defe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:18.066749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b69c1-4400-42c3-afdc-c60cffe9defe None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:18.067026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b69c1-4400-42c3-afdc-c60cffe9defe None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:18.067296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b69c1-4400-42c3-afdc-c60cffe9defe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:18.076264 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9233c-46bc-4856-a9c2-96f4c9744bfb None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:18.077555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b69c1-4400-42c3-afdc-c60cffe9defe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:18.079251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b69c1-4400-42c3-afdc-c60cffe9defe None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-9b3b69c1-4400-42c3-afdc-c60cffe9defe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:14:18.096472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:18.097500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:18.098578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:14:18.098578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:14:18.098578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:18.098578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:18.100422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 546/1083] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:14:18.106494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b69c1-4400-42c3-afdc-c60cffe9defe None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:18.109216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cd664-8e8e-48ef-876d-7330cba1264c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:18.109894 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd664-8e8e-48ef-876d-7330cba1264c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:18.110178 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd664-8e8e-48ef-876d-7330cba1264c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:18.110456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd664-8e8e-48ef-876d-7330cba1264c None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:18.111294 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cd664-8e8e-48ef-876d-7330cba1264c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:18.111604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cd664-8e8e-48ef-876d-7330cba1264c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:18.113059 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cd664-8e8e-48ef-876d-7330cba1264c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:18.130597 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 538/1084] 10.222.0.44 () {56 vars in 1133 bytes} [Mon May 9 04:14:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:18.136155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cd664-8e8e-48ef-876d-7330cba1264c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:18.202359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b69c1-4400-42c3-afdc-c60cffe9defe None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:18.229265 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cd664-8e8e-48ef-876d-7330cba1264c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:18.234201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22481947-18cc-48ac-8ad8-0f7af80faf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:18.234856 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22481947-18cc-48ac-8ad8-0f7af80faf85 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:18.235135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22481947-18cc-48ac-8ad8-0f7af80faf85 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:18.235408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22481947-18cc-48ac-8ad8-0f7af80faf85 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:18.236146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22481947-18cc-48ac-8ad8-0f7af80faf85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:18.236452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22481947-18cc-48ac-8ad8-0f7af80faf85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:18.237803 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22481947-18cc-48ac-8ad8-0f7af80faf85 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:18.249228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 547/1085] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:18.262167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e78be-3254-410c-97b3-eda86d946a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:18.263347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e78be-3254-410c-97b3-eda86d946a2b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:18.263788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e78be-3254-410c-97b3-eda86d946a2b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:18.264121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e78be-3254-410c-97b3-eda86d946a2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:18.335292 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e78be-3254-410c-97b3-eda86d946a2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:18.336792 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e78be-3254-410c-97b3-eda86d946a2b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:18.456815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 539/1086] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:14:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:14:18.462212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22481947-18cc-48ac-8ad8-0f7af80faf85 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:18.563671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22481947-18cc-48ac-8ad8-0f7af80faf85 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:18.597297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783732e8-9b0c-4fd3-b2b3-513138ee09bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:18.598082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783732e8-9b0c-4fd3-b2b3-513138ee09bd None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:18.598361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783732e8-9b0c-4fd3-b2b3-513138ee09bd None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:18.598657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783732e8-9b0c-4fd3-b2b3-513138ee09bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:18.608628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783732e8-9b0c-4fd3-b2b3-513138ee09bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:18.610063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783732e8-9b0c-4fd3-b2b3-513138ee09bd None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-783732e8-9b0c-4fd3-b2b3-513138ee09bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:14:18.626419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:18.627338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:18.628438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:14:18.628438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:14:18.628438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:14:18.628438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:18.628438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 548/1087] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:18.633176 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e78be-3254-410c-97b3-eda86d946a2b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:18.726526 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e78be-3254-410c-97b3-eda86d946a2b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:18.761116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c4087-b698-414a-8cba-3562fa05a2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:18.761841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c4087-b698-414a-8cba-3562fa05a2bb None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:18.762158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c4087-b698-414a-8cba-3562fa05a2bb None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:18.762962 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c4087-b698-414a-8cba-3562fa05a2bb None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:18.763773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c4087-b698-414a-8cba-3562fa05a2bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:18.764107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c4087-b698-414a-8cba-3562fa05a2bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:18.765460 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c4087-b698-414a-8cba-3562fa05a2bb None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:18.776386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 540/1088] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:18.782102 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783732e8-9b0c-4fd3-b2b3-513138ee09bd None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:18.881856 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783732e8-9b0c-4fd3-b2b3-513138ee09bd None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:18.913637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7562f-bb7f-445f-8df5-24dc27179c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:18.914431 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7562f-bb7f-445f-8df5-24dc27179c1e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:18.914714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7562f-bb7f-445f-8df5-24dc27179c1e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:18.914992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7562f-bb7f-445f-8df5-24dc27179c1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:18.921714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7562f-bb7f-445f-8df5-24dc27179c1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:18.923148 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7562f-bb7f-445f-8df5-24dc27179c1e None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-dcf7562f-bb7f-445f-8df5-24dc27179c1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:14:18.937470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:18.938525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:18.939454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:14:18.939454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:14:18.939454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:14:18.939454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:14:18.939454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:14:18.939454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:14:18.939454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:14:18.939454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:18.939454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 549/1089] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:14:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:18.944431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c4087-b698-414a-8cba-3562fa05a2bb None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:18.976619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23932018-b5ef-423d-ad3d-4b6e3e32d65d None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:18.976834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23932018-b5ef-423d-ad3d-4b6e3e32d65d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:18.977114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23932018-b5ef-423d-ad3d-4b6e3e32d65d None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:18.978479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 550/1090] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:14:18.983376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492c4a7-ba9c-4c98-b806-e4a569171fd7 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:18.983666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492c4a7-ba9c-4c98-b806-e4a569171fd7 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:18.983973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492c4a7-ba9c-4c98-b806-e4a569171fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:19.000792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-a492c4a7-ba9c-4c98-b806-e4a569171fd7 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:19.001975 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a492c4a7-ba9c-4c98-b806-e4a569171fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:19.037958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c4087-b698-414a-8cba-3562fa05a2bb None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:19.064297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 551/1091] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:14:19.069678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d694e23-061f-4f3c-b8ec-f55455ae3d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:19.070330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d694e23-061f-4f3c-b8ec-f55455ae3d43 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:19.070611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d694e23-061f-4f3c-b8ec-f55455ae3d43 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:19.070883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d694e23-061f-4f3c-b8ec-f55455ae3d43 None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:19.071655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d694e23-061f-4f3c-b8ec-f55455ae3d43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:19.071992 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d694e23-061f-4f3c-b8ec-f55455ae3d43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:19.073390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d694e23-061f-4f3c-b8ec-f55455ae3d43 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:19.085949 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 541/1092] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:14:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:19.087956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492c4a7-ba9c-4c98-b806-e4a569171fd7 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:19.092210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d694e23-061f-4f3c-b8ec-f55455ae3d43 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:19.181032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492c4a7-ba9c-4c98-b806-e4a569171fd7 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:19.184130 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d694e23-061f-4f3c-b8ec-f55455ae3d43 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:19.213039 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d7bde-7461-44db-a178-c704b761f7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:19.213832 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d7bde-7461-44db-a178-c704b761f7ab None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:19.214117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d7bde-7461-44db-a178-c704b761f7ab None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:19.214392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d7bde-7461-44db-a178-c704b761f7ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:19.216673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2271c-c8c3-48e3-9ede-38cf3e96cf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:19.217317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2271c-c8c3-48e3-9ede-38cf3e96cf81 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:19.217597 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2271c-c8c3-48e3-9ede-38cf3e96cf81 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:19.217868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2271c-c8c3-48e3-9ede-38cf3e96cf81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:19.218688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2271c-c8c3-48e3-9ede-38cf3e96cf81 None admin] RBAC: Adding query filter params (role.id=7de5200fcd0943d9834b6e12861d5941, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=faf6107735424ff7901614953e7df403) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:19.218999 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2271c-c8c3-48e3-9ede-38cf3e96cf81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:19.220443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2271c-c8c3-48e3-9ede-38cf3e96cf81 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:19.224566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d7bde-7461-44db-a178-c704b761f7ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:14:19.225986 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 542/1093] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:14:19 2022] GET /identity/v3/role_assignments?user.id=faf6107735424ff7901614953e7df403&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=7de5200fcd0943d9834b6e12861d5941 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-5e6d7bde-7461-44db-a178-c704b761f7ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:14:19.232763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:19.233888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:19.234837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:19.234837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:19.234837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:19.234837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:19.234837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:14:19.234837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:14:19.234837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:14:19.234837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:14:19.234837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:14:19.234837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:14:19.234837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:14:19.234837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:19.234837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 552/1094] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:14:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:19.240461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2271c-c8c3-48e3-9ede-38cf3e96cf81 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:19.341880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2271c-c8c3-48e3-9ede-38cf3e96cf81 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:19.375667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c94086-fbb8-4f2f-8a63-92b90ca14d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:19.376374 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c94086-fbb8-4f2f-8a63-92b90ca14d1e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:19.376657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c94086-fbb8-4f2f-8a63-92b90ca14d1e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:19.376933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c94086-fbb8-4f2f-8a63-92b90ca14d1e None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:19.377693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c94086-fbb8-4f2f-8a63-92b90ca14d1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:19.378001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c94086-fbb8-4f2f-8a63-92b90ca14d1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:19.379395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c94086-fbb8-4f2f-8a63-92b90ca14d1e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:19.393488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 543/1095] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:14:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:19.395058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d7bde-7461-44db-a178-c704b761f7ab None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:19.491129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d7bde-7461-44db-a178-c704b761f7ab None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:19.524755 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6068298-a6d0-49d8-afcb-6217db25f09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:19.525652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6068298-a6d0-49d8-afcb-6217db25f09d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:19.525929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6068298-a6d0-49d8-afcb-6217db25f09d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:19.526203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6068298-a6d0-49d8-afcb-6217db25f09d None admin] PATH_INFO: `/v3/users/d0c6e49d332744289bde848c528bf66c` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:19.527383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6068298-a6d0-49d8-afcb-6217db25f09d None admin] RBAC: Authorizing `identity:get_user(user_id=d0c6e49d332744289bde848c528bf66c)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:19.529393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6068298-a6d0-49d8-afcb-6217db25f09d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:19.531014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 553/1096] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:14:19 2022] GET /identity/v3/users/d0c6e49d332744289bde848c528bf66c => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:19.536903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c94086-fbb8-4f2f-8a63-92b90ca14d1e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:19.634035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c94086-fbb8-4f2f-8a63-92b90ca14d1e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:19.667441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239a823-3be4-459f-aad0-aac463b65dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:19.668945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239a823-3be4-459f-aad0-aac463b65dac None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:19.669239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239a823-3be4-459f-aad0-aac463b65dac None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:19.669513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239a823-3be4-459f-aad0-aac463b65dac None admin] PATH_INFO: `/v3/projects/eb5f521580e74518821f8f0072d10858` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:19.670700 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239a823-3be4-459f-aad0-aac463b65dac None admin] RBAC: Authorizing `identity:get_project(project_id=eb5f521580e74518821f8f0072d10858)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:19.672200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239a823-3be4-459f-aad0-aac463b65dac None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:19.674036 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 544/1097] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:14:19 2022] GET /identity/v3/projects/eb5f521580e74518821f8f0072d10858 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:19.679998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6068298-a6d0-49d8-afcb-6217db25f09d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:19.780359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6068298-a6d0-49d8-afcb-6217db25f09d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:19.814127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e9fb8-1729-4c65-a231-12ebda016c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:19.815279 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e9fb8-1729-4c65-a231-12ebda016c39 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:19.815558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e9fb8-1729-4c65-a231-12ebda016c39 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:19.815837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e9fb8-1729-4c65-a231-12ebda016c39 None admin] PATH_INFO: `/v3/projects/eb5f521580e74518821f8f0072d10858/users/d0c6e49d332744289bde848c528bf66c/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:19.818994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e9fb8-1729-4c65-a231-12ebda016c39 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb5f521580e74518821f8f0072d10858, user_id=d0c6e49d332744289bde848c528bf66c, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:19.820472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e9fb8-1729-4c65-a231-12ebda016c39 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:19.835391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 554/1098] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:14:19 2022] PUT /identity/v3/projects/eb5f521580e74518821f8f0072d10858/users/d0c6e49d332744289bde848c528bf66c/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:14:22.049221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48d758-129f-4612-9441-4d21a207288e None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:22.049768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48d758-129f-4612-9441-4d21a207288e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:22.049768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48d758-129f-4612-9441-4d21a207288e None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:22.051028 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 545/1099] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:22.056445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d48ecf-b74b-4b50-b977-e6ae207672e4 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:22.056883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d48ecf-b74b-4b50-b977-e6ae207672e4 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:22.057006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d48ecf-b74b-4b50-b977-e6ae207672e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:22.073598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-32d48ecf-b74b-4b50-b977-e6ae207672e4 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:22.075331 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-32d48ecf-b74b-4b50-b977-e6ae207672e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:22.137168 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 555/1100] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:22.159985 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48d758-129f-4612-9441-4d21a207288e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:22.255871 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48d758-129f-4612-9441-4d21a207288e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:22.285958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08232662-4c2c-4a1a-83e0-352bf699b74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:22.286728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08232662-4c2c-4a1a-83e0-352bf699b74d None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:22.287012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08232662-4c2c-4a1a-83e0-352bf699b74d None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:22.287281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08232662-4c2c-4a1a-83e0-352bf699b74d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:22.297191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08232662-4c2c-4a1a-83e0-352bf699b74d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-08232662-4c2c-4a1a-83e0-352bf699b74d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:14:22.304718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:22.305891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:22.307016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:22.307016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:22.307016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:22.307016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:22.307016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:14:22.307016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:14:22.307016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:14:22.307016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:14:22.307016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:14:22.307016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:14:22.307016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:14:22.307016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:22.307016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 546/1101] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:14:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:22.312498 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d48ecf-b74b-4b50-b977-e6ae207672e4 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:22.404063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d48ecf-b74b-4b50-b977-e6ae207672e4 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:22.435821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3fbd1-40d9-4b57-8bfe-b12fea82e241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:22.436501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3fbd1-40d9-4b57-8bfe-b12fea82e241 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:22.436781 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3fbd1-40d9-4b57-8bfe-b12fea82e241 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:22.437052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3fbd1-40d9-4b57-8bfe-b12fea82e241 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:22.437790 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3fbd1-40d9-4b57-8bfe-b12fea82e241 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:22.438092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3fbd1-40d9-4b57-8bfe-b12fea82e241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:22.439443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3fbd1-40d9-4b57-8bfe-b12fea82e241 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:22.446950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 556/1102] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:14:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:22.452957 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08232662-4c2c-4a1a-83e0-352bf699b74d None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:22.545082 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08232662-4c2c-4a1a-83e0-352bf699b74d None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:22.575748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31250eff-bb7e-44ba-b1b9-8b3d84f0fede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:22.576614 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31250eff-bb7e-44ba-b1b9-8b3d84f0fede None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:22.576894 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31250eff-bb7e-44ba-b1b9-8b3d84f0fede None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:22.577167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31250eff-bb7e-44ba-b1b9-8b3d84f0fede None admin] PATH_INFO: `/v3/users/d0c6e49d332744289bde848c528bf66c` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:22.578324 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31250eff-bb7e-44ba-b1b9-8b3d84f0fede None admin] RBAC: Authorizing `identity:get_user(user_id=d0c6e49d332744289bde848c528bf66c)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:22.580009 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31250eff-bb7e-44ba-b1b9-8b3d84f0fede None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:22.581583 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 547/1103] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:14:22 2022] GET /identity/v3/users/d0c6e49d332744289bde848c528bf66c => generated 285 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:22.587212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3fbd1-40d9-4b57-8bfe-b12fea82e241 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:22.679555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3fbd1-40d9-4b57-8bfe-b12fea82e241 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:22.710111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047f61b-a786-46f5-9131-c005411262f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:22.710740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047f61b-a786-46f5-9131-c005411262f9 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:22.711020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047f61b-a786-46f5-9131-c005411262f9 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:22.711293 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047f61b-a786-46f5-9131-c005411262f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:22.775482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047f61b-a786-46f5-9131-c005411262f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:22.776965 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047f61b-a786-46f5-9131-c005411262f9 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:22.868060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 557/1104] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:14:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:14:22.873976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31250eff-bb7e-44ba-b1b9-8b3d84f0fede None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:22.966466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31250eff-bb7e-44ba-b1b9-8b3d84f0fede None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:22.997047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab8bf8-8d28-4418-af83-5f4dcd7b98c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:22.997879 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab8bf8-8d28-4418-af83-5f4dcd7b98c4 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:22.998158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab8bf8-8d28-4418-af83-5f4dcd7b98c4 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:22.998428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab8bf8-8d28-4418-af83-5f4dcd7b98c4 None admin] PATH_INFO: `/v3/projects/eb5f521580e74518821f8f0072d10858` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:22.999576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab8bf8-8d28-4418-af83-5f4dcd7b98c4 None admin] RBAC: Authorizing `identity:get_project(project_id=eb5f521580e74518821f8f0072d10858)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:23.001024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab8bf8-8d28-4418-af83-5f4dcd7b98c4 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:23.002643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 548/1105] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:14:22 2022] GET /identity/v3/projects/eb5f521580e74518821f8f0072d10858 => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:23.008971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047f61b-a786-46f5-9131-c005411262f9 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:23.103422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047f61b-a786-46f5-9131-c005411262f9 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:23.135240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b780db9-df0c-4bbd-b8f3-dc1a8b56171b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:23.135828 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b780db9-df0c-4bbd-b8f3-dc1a8b56171b None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:23.136144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b780db9-df0c-4bbd-b8f3-dc1a8b56171b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:23.136424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b780db9-df0c-4bbd-b8f3-dc1a8b56171b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:23.138188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b780db9-df0c-4bbd-b8f3-dc1a8b56171b None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=eb5f521580e74518821f8f0072d10858, user.id=d0c6e49d332744289bde848c528bf66c) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:23.138507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b780db9-df0c-4bbd-b8f3-dc1a8b56171b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:23.139956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b780db9-df0c-4bbd-b8f3-dc1a8b56171b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:23.145510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 558/1106] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:14:23 2022] GET /identity/v3/role_assignments?user.id=d0c6e49d332744289bde848c528bf66c&scope.project.id=eb5f521580e74518821f8f0072d10858&role.id=e59113c0366049e889df3f55eed90c72 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:25.361912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac533ee-57e6-4ea4-807e-24a82b5209bb None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:25.362439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac533ee-57e6-4ea4-807e-24a82b5209bb None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:25.362439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac533ee-57e6-4ea4-807e-24a82b5209bb None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:25.363799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 549/1107] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:25.369014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ded8e-b563-4ec5-b7fb-4f4e7d8fe377 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:25.369400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ded8e-b563-4ec5-b7fb-4f4e7d8fe377 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:25.369568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ded8e-b563-4ec5-b7fb-4f4e7d8fe377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:25.385962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-445ded8e-b563-4ec5-b7fb-4f4e7d8fe377 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:25.387133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-445ded8e-b563-4ec5-b7fb-4f4e7d8fe377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:25.448191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 559/1108] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:25.472990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac533ee-57e6-4ea4-807e-24a82b5209bb None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:25.567121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac533ee-57e6-4ea4-807e-24a82b5209bb None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:25.598570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b5136-198c-4add-ab6e-c135a2a928f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:25.599349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b5136-198c-4add-ab6e-c135a2a928f4 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:25.599632 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b5136-198c-4add-ab6e-c135a2a928f4 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:25.599926 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b5136-198c-4add-ab6e-c135a2a928f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:25.601072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b5136-198c-4add-ab6e-c135a2a928f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:25.602492 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b5136-198c-4add-ab6e-c135a2a928f4 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:25.604085 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 550/1109] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:25.609843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ded8e-b563-4ec5-b7fb-4f4e7d8fe377 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:25.701885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ded8e-b563-4ec5-b7fb-4f4e7d8fe377 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:25.733847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25135e7-432e-44b0-9d3e-0925c8b99a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:25.734503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25135e7-432e-44b0-9d3e-0925c8b99a52 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:25.734786 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25135e7-432e-44b0-9d3e-0925c8b99a52 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:25.735086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25135e7-432e-44b0-9d3e-0925c8b99a52 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:25.735796 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25135e7-432e-44b0-9d3e-0925c8b99a52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:25.737260 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25135e7-432e-44b0-9d3e-0925c8b99a52 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:25.757807 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 560/1110] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:14:25 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:14:27.989914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad74a3-aaa8-4a58-b922-03a3c1818b27 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:27.990420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad74a3-aaa8-4a58-b922-03a3c1818b27 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:27.990533 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad74a3-aaa8-4a58-b922-03a3c1818b27 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:27.991800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 551/1111] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:27.997677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924927d-b201-4aa6-a29d-3f301b0ff4df None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:27.998095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924927d-b201-4aa6-a29d-3f301b0ff4df None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:27.998230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924927d-b201-4aa6-a29d-3f301b0ff4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:28.015236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-1924927d-b201-4aa6-a29d-3f301b0ff4df None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:28.016388 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1924927d-b201-4aa6-a29d-3f301b0ff4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:28.085859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 561/1112] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:28.109059 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad74a3-aaa8-4a58-b922-03a3c1818b27 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:28.205100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad74a3-aaa8-4a58-b922-03a3c1818b27 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:28.236022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787ae55-ecf4-4b69-a7e0-7885ed1e58a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:28.236864 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787ae55-ecf4-4b69-a7e0-7885ed1e58a0 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:28.237171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787ae55-ecf4-4b69-a7e0-7885ed1e58a0 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:28.237444 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787ae55-ecf4-4b69-a7e0-7885ed1e58a0 None admin] PATH_INFO: `/v3/roles/7600100dce614d9bb4d47b1e660b7bb2` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:28.238876 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787ae55-ecf4-4b69-a7e0-7885ed1e58a0 None admin] RBAC: Authorizing `identity:get_role(role_id=7600100dce614d9bb4d47b1e660b7bb2)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:28.240250 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787ae55-ecf4-4b69-a7e0-7885ed1e58a0 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:28.241594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 552/1113] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:14:28 2022] GET /identity/v3/roles/7600100dce614d9bb4d47b1e660b7bb2 => generated 223 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:28.248072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924927d-b201-4aa6-a29d-3f301b0ff4df None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:28.339481 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924927d-b201-4aa6-a29d-3f301b0ff4df None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:28.370490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50045c-c307-4e4b-9b81-41e98c558c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:28.371424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50045c-c307-4e4b-9b81-41e98c558c29 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:28.371737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50045c-c307-4e4b-9b81-41e98c558c29 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:28.372044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50045c-c307-4e4b-9b81-41e98c558c29 None admin] PATH_INFO: `/v3/users/511ec5a356f64998b2896e8343287f1e` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:28.384176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50045c-c307-4e4b-9b81-41e98c558c29 None admin] RBAC: Authorizing `identity:get_user(user_id=511ec5a356f64998b2896e8343287f1e)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:28.385592 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50045c-c307-4e4b-9b81-41e98c558c29 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:28.387810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 562/1114] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:14:28 2022] GET /identity/v3/users/511ec5a356f64998b2896e8343287f1e => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:28.393461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787ae55-ecf4-4b69-a7e0-7885ed1e58a0 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:28.493373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787ae55-ecf4-4b69-a7e0-7885ed1e58a0 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:28.526952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c986f-5d42-408c-b080-7c2998ea17e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:28.527657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c986f-5d42-408c-b080-7c2998ea17e7 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:28.527975 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c986f-5d42-408c-b080-7c2998ea17e7 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:28.528258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c986f-5d42-408c-b080-7c2998ea17e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:28.591430 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c986f-5d42-408c-b080-7c2998ea17e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:28.592831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c986f-5d42-408c-b080-7c2998ea17e7 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:28.683125 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 553/1115] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:14:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:14:28.689577 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50045c-c307-4e4b-9b81-41e98c558c29 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:28.782052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50045c-c307-4e4b-9b81-41e98c558c29 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:28.813236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1805186-ac38-4e9c-9ac5-4a36ae5b7e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:28.814081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1805186-ac38-4e9c-9ac5-4a36ae5b7e44 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:28.814356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1805186-ac38-4e9c-9ac5-4a36ae5b7e44 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:28.814629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1805186-ac38-4e9c-9ac5-4a36ae5b7e44 None admin] PATH_INFO: `/v3/projects/eb5f521580e74518821f8f0072d10858` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:28.815777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1805186-ac38-4e9c-9ac5-4a36ae5b7e44 None admin] RBAC: Authorizing `identity:get_project(project_id=eb5f521580e74518821f8f0072d10858)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:28.817241 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1805186-ac38-4e9c-9ac5-4a36ae5b7e44 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:28.818968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 563/1116] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:14:28 2022] GET /identity/v3/projects/eb5f521580e74518821f8f0072d10858 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:28.825854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c986f-5d42-408c-b080-7c2998ea17e7 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:28.916246 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c986f-5d42-408c-b080-7c2998ea17e7 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:28.947884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6d5d6-aa6c-4a68-92f9-af0928aeec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:28.948549 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6d5d6-aa6c-4a68-92f9-af0928aeec05 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:28.948827 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6d5d6-aa6c-4a68-92f9-af0928aeec05 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:28.949096 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6d5d6-aa6c-4a68-92f9-af0928aeec05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:28.949945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6d5d6-aa6c-4a68-92f9-af0928aeec05 None admin] RBAC: Adding query filter params (role.id=7600100dce614d9bb4d47b1e660b7bb2, scope.project.id=eb5f521580e74518821f8f0072d10858, user.id=511ec5a356f64998b2896e8343287f1e) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:28.950249 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6d5d6-aa6c-4a68-92f9-af0928aeec05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:28.951648 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6d5d6-aa6c-4a68-92f9-af0928aeec05 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:28.957017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 554/1117] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:14:28 2022] GET /identity/v3/role_assignments?user.id=511ec5a356f64998b2896e8343287f1e&scope.project.id=eb5f521580e74518821f8f0072d10858&role.id=7600100dce614d9bb4d47b1e660b7bb2 => generated 259 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:31.118871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d53c8-1b0d-4b85-b2e8-4e6f3e0c9b5d None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:31.119389 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d53c8-1b0d-4b85-b2e8-4e6f3e0c9b5d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:31.119389 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d53c8-1b0d-4b85-b2e8-4e6f3e0c9b5d None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:31.120783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 564/1118] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:31.126494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b69ab5-dc9e-4ed8-a433-bd1c06ee58c5 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:31.126930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b69ab5-dc9e-4ed8-a433-bd1c06ee58c5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:31.127049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b69ab5-dc9e-4ed8-a433-bd1c06ee58c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:31.143441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d7b69ab5-dc9e-4ed8-a433-bd1c06ee58c5 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:31.144621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d7b69ab5-dc9e-4ed8-a433-bd1c06ee58c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:31.206145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 555/1119] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:31.229082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d53c8-1b0d-4b85-b2e8-4e6f3e0c9b5d None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:31.322907 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d53c8-1b0d-4b85-b2e8-4e6f3e0c9b5d None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:31.355624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1eb44-e7be-45e0-8e46-8b51b1f978d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:31.356648 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1eb44-e7be-45e0-8e46-8b51b1f978d1 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:31.356932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1eb44-e7be-45e0-8e46-8b51b1f978d1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:31.357211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1eb44-e7be-45e0-8e46-8b51b1f978d1 None admin] PATH_INFO: `/v3/roles/7600100dce614d9bb4d47b1e660b7bb2` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:31.358680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1eb44-e7be-45e0-8e46-8b51b1f978d1 None admin] RBAC: Authorizing `identity:get_role(role_id=7600100dce614d9bb4d47b1e660b7bb2)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:31.360061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1eb44-e7be-45e0-8e46-8b51b1f978d1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:31.361554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 565/1120] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:14:31 2022] GET /identity/v3/roles/7600100dce614d9bb4d47b1e660b7bb2 => generated 223 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:31.367337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b69ab5-dc9e-4ed8-a433-bd1c06ee58c5 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:31.458991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b69ab5-dc9e-4ed8-a433-bd1c06ee58c5 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:31.490177 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d46d242-4d63-495a-b0f9-255deecd5300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:31.491031 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46d242-4d63-495a-b0f9-255deecd5300 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:31.491313 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46d242-4d63-495a-b0f9-255deecd5300 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:31.491586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46d242-4d63-495a-b0f9-255deecd5300 None admin] PATH_INFO: `/v3/users/511ec5a356f64998b2896e8343287f1e` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:31.492769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d46d242-4d63-495a-b0f9-255deecd5300 None admin] RBAC: Authorizing `identity:get_user(user_id=511ec5a356f64998b2896e8343287f1e)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:31.494173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d46d242-4d63-495a-b0f9-255deecd5300 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:31.495743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 556/1121] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:14:31 2022] GET /identity/v3/users/511ec5a356f64998b2896e8343287f1e => generated 286 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:31.502026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1eb44-e7be-45e0-8e46-8b51b1f978d1 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:31.598308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1eb44-e7be-45e0-8e46-8b51b1f978d1 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:31.630191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6d2a9-792e-45ca-a78a-87e362825e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:31.631034 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6d2a9-792e-45ca-a78a-87e362825e09 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:31.631316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6d2a9-792e-45ca-a78a-87e362825e09 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:31.631591 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6d2a9-792e-45ca-a78a-87e362825e09 None admin] PATH_INFO: `/v3/projects/eb5f521580e74518821f8f0072d10858` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:31.632785 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6d2a9-792e-45ca-a78a-87e362825e09 None admin] RBAC: Authorizing `identity:get_project(project_id=eb5f521580e74518821f8f0072d10858)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:31.634201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6d2a9-792e-45ca-a78a-87e362825e09 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:31.635861 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 566/1122] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:14:31 2022] GET /identity/v3/projects/eb5f521580e74518821f8f0072d10858 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:31.642350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d46d242-4d63-495a-b0f9-255deecd5300 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:31.733637 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d46d242-4d63-495a-b0f9-255deecd5300 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:31.765776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f1060-d331-4d3b-ac66-d52cf55a82e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:31.766657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f1060-d331-4d3b-ac66-d52cf55a82e0 None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:31.766933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f1060-d331-4d3b-ac66-d52cf55a82e0 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:31.767202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f1060-d331-4d3b-ac66-d52cf55a82e0 None admin] PATH_INFO: `/v3/projects/eb5f521580e74518821f8f0072d10858/users/511ec5a356f64998b2896e8343287f1e/roles/7600100dce614d9bb4d47b1e660b7bb2` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:31.769332 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f1060-d331-4d3b-ac66-d52cf55a82e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb5f521580e74518821f8f0072d10858, user_id=511ec5a356f64998b2896e8343287f1e, role_id=7600100dce614d9bb4d47b1e660b7bb2)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:31.770735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f1060-d331-4d3b-ac66-d52cf55a82e0 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:31.782846 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 557/1123] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:14:31 2022] PUT /identity/v3/projects/eb5f521580e74518821f8f0072d10858/users/511ec5a356f64998b2896e8343287f1e/roles/7600100dce614d9bb4d47b1e660b7bb2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:14:33.972223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9afc12-8186-4b38-a8ca-94145cb1368d None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:33.972769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9afc12-8186-4b38-a8ca-94145cb1368d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:33.972769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9afc12-8186-4b38-a8ca-94145cb1368d None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:33.974126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 567/1124] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:33.979280 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6b55f-2075-459f-8001-eb2bc6ce9c8e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:33.979671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6b55f-2075-459f-8001-eb2bc6ce9c8e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:33.979839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6b55f-2075-459f-8001-eb2bc6ce9c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:33.996696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-66d6b55f-2075-459f-8001-eb2bc6ce9c8e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:33.998095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-66d6b55f-2075-459f-8001-eb2bc6ce9c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:34.059630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 558/1125] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:34.090169 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9afc12-8186-4b38-a8ca-94145cb1368d None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:34.186549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9afc12-8186-4b38-a8ca-94145cb1368d None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:34.217234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318ff8a-0d48-442e-b4f1-166f671fac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:34.218074 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318ff8a-0d48-442e-b4f1-166f671fac49 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:34.218348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318ff8a-0d48-442e-b4f1-166f671fac49 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:34.218615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318ff8a-0d48-442e-b4f1-166f671fac49 None admin] PATH_INFO: `/v3/roles/7600100dce614d9bb4d47b1e660b7bb2` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:34.220077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318ff8a-0d48-442e-b4f1-166f671fac49 None admin] RBAC: Authorizing `identity:get_role(role_id=7600100dce614d9bb4d47b1e660b7bb2)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:34.221434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318ff8a-0d48-442e-b4f1-166f671fac49 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:34.222778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 568/1126] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:14:34 2022] GET /identity/v3/roles/7600100dce614d9bb4d47b1e660b7bb2 => generated 223 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:34.228613 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6b55f-2075-459f-8001-eb2bc6ce9c8e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:34.321094 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6b55f-2075-459f-8001-eb2bc6ce9c8e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:34.353259 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0075de4-c934-4649-b932-5cd9fdcbf30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:34.354108 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0075de4-c934-4649-b932-5cd9fdcbf30c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:34.354386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0075de4-c934-4649-b932-5cd9fdcbf30c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:34.354656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0075de4-c934-4649-b932-5cd9fdcbf30c None admin] PATH_INFO: `/v3/users/511ec5a356f64998b2896e8343287f1e` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:34.355854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0075de4-c934-4649-b932-5cd9fdcbf30c None admin] RBAC: Authorizing `identity:get_user(user_id=511ec5a356f64998b2896e8343287f1e)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:34.357297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0075de4-c934-4649-b932-5cd9fdcbf30c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:34.358886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 559/1127] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:14:34 2022] GET /identity/v3/users/511ec5a356f64998b2896e8343287f1e => generated 286 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:34.364586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318ff8a-0d48-442e-b4f1-166f671fac49 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:34.457886 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318ff8a-0d48-442e-b4f1-166f671fac49 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:34.489112 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ce85f-0ede-4bed-9f7d-57d6c9a22eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:34.489740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce85f-0ede-4bed-9f7d-57d6c9a22eac None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:34.490016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce85f-0ede-4bed-9f7d-57d6c9a22eac None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:34.490285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ce85f-0ede-4bed-9f7d-57d6c9a22eac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:34.551557 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ce85f-0ede-4bed-9f7d-57d6c9a22eac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:34.553046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ce85f-0ede-4bed-9f7d-57d6c9a22eac None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:34.643999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 569/1128] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:14:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:14:34.649982 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0075de4-c934-4649-b932-5cd9fdcbf30c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:34.741970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0075de4-c934-4649-b932-5cd9fdcbf30c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:34.774820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f47bb-bdc3-4fe4-b866-09af84cf1831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:34.775666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f47bb-bdc3-4fe4-b866-09af84cf1831 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:34.775970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f47bb-bdc3-4fe4-b866-09af84cf1831 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:34.776240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f47bb-bdc3-4fe4-b866-09af84cf1831 None admin] PATH_INFO: `/v3/projects/eb5f521580e74518821f8f0072d10858` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:34.777434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f47bb-bdc3-4fe4-b866-09af84cf1831 None admin] RBAC: Authorizing `identity:get_project(project_id=eb5f521580e74518821f8f0072d10858)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:34.778904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f47bb-bdc3-4fe4-b866-09af84cf1831 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:34.780581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 560/1129] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:14:34 2022] GET /identity/v3/projects/eb5f521580e74518821f8f0072d10858 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:34.786805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ce85f-0ede-4bed-9f7d-57d6c9a22eac None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:34.879160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ce85f-0ede-4bed-9f7d-57d6c9a22eac None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:34.909977 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401387d-69fa-4e96-8174-3d9995f60071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:34.910557 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401387d-69fa-4e96-8174-3d9995f60071 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:34.910837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401387d-69fa-4e96-8174-3d9995f60071 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:34.911112 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401387d-69fa-4e96-8174-3d9995f60071 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:34.911955 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401387d-69fa-4e96-8174-3d9995f60071 None admin] RBAC: Adding query filter params (role.id=7600100dce614d9bb4d47b1e660b7bb2, scope.project.id=eb5f521580e74518821f8f0072d10858, user.id=511ec5a356f64998b2896e8343287f1e) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:34.912267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401387d-69fa-4e96-8174-3d9995f60071 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:34.913677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401387d-69fa-4e96-8174-3d9995f60071 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:34.919657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 570/1130] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:14:34 2022] GET /identity/v3/role_assignments?user.id=511ec5a356f64998b2896e8343287f1e&scope.project.id=eb5f521580e74518821f8f0072d10858&role.id=7600100dce614d9bb4d47b1e660b7bb2 => generated 608 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:37.135396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0c079-4030-4e4b-94e0-df4e9e7214d2 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:37.135933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0c079-4030-4e4b-94e0-df4e9e7214d2 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:37.135933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0c079-4030-4e4b-94e0-df4e9e7214d2 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:37.137395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 561/1131] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:37.142769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3e736-f9be-4c7c-9212-c98edb794c3e None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:37.143173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3e736-f9be-4c7c-9212-c98edb794c3e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:37.143328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3e736-f9be-4c7c-9212-c98edb794c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:37.159452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2cb3e736-f9be-4c7c-9212-c98edb794c3e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:37.160652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2cb3e736-f9be-4c7c-9212-c98edb794c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:37.222352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 571/1132] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:37.245745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0c079-4030-4e4b-94e0-df4e9e7214d2 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:37.339773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0c079-4030-4e4b-94e0-df4e9e7214d2 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:37.371245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f41867-773d-4eeb-851d-9803b3796312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:37.372184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f41867-773d-4eeb-851d-9803b3796312 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:37.372467 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f41867-773d-4eeb-851d-9803b3796312 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:37.372741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f41867-773d-4eeb-851d-9803b3796312 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:37.373893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f41867-773d-4eeb-851d-9803b3796312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:37.375301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f41867-773d-4eeb-851d-9803b3796312 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:37.376906 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 562/1133] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:37.382530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3e736-f9be-4c7c-9212-c98edb794c3e None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:37.474847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3e736-f9be-4c7c-9212-c98edb794c3e None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:37.506690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44dc2d-900e-4156-a557-478692e3060f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:37.507333 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44dc2d-900e-4156-a557-478692e3060f None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:37.507616 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44dc2d-900e-4156-a557-478692e3060f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:37.507920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44dc2d-900e-4156-a557-478692e3060f None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:37.508626 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44dc2d-900e-4156-a557-478692e3060f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:37.510065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44dc2d-900e-4156-a557-478692e3060f None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:37.533917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 572/1134] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:14:37 2022] POST /identity/v3/projects => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:14:39.763591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4300bc-0890-45ac-b272-556e096b6efe None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:39.764076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4300bc-0890-45ac-b272-556e096b6efe None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:39.764170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4300bc-0890-45ac-b272-556e096b6efe None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:39.765497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 563/1135] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:39.770839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779021-b56a-4351-97c4-086e79191932 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:39.771246 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779021-b56a-4351-97c4-086e79191932 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:39.771406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae779021-b56a-4351-97c4-086e79191932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:39.788182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-ae779021-b56a-4351-97c4-086e79191932 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:39.789358 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ae779021-b56a-4351-97c4-086e79191932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:39.851024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 573/1136] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:39.875252 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4300bc-0890-45ac-b272-556e096b6efe None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:39.969264 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4300bc-0890-45ac-b272-556e096b6efe None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:40.000587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b43c8-111b-42f0-a6f7-e524f760e633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:40.001376 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b43c8-111b-42f0-a6f7-e524f760e633 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:40.001656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b43c8-111b-42f0-a6f7-e524f760e633 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:40.001926 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b43c8-111b-42f0-a6f7-e524f760e633 None admin] PATH_INFO: `/v3/domains/default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:40.003070 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b43c8-111b-42f0-a6f7-e524f760e633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:40.004510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b43c8-111b-42f0-a6f7-e524f760e633 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:40.006073 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 564/1137] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:14:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:40.011603 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae779021-b56a-4351-97c4-086e79191932 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:40.103199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae779021-b56a-4351-97c4-086e79191932 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:40.133987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b7ec2-fe57-43f1-a892-1caedcbc63f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:40.134639 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b7ec2-fe57-43f1-a892-1caedcbc63f1 None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:40.134920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b7ec2-fe57-43f1-a892-1caedcbc63f1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:40.135191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b7ec2-fe57-43f1-a892-1caedcbc63f1 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:40.135885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b7ec2-fe57-43f1-a892-1caedcbc63f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:40.138092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b7ec2-fe57-43f1-a892-1caedcbc63f1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:40.156970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 574/1138] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:14:40 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:14:42.388450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c4923-21fe-4d12-93fb-a324ab260a44 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:42.389010 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c4923-21fe-4d12-93fb-a324ab260a44 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:42.389010 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c4923-21fe-4d12-93fb-a324ab260a44 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:42.390331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 565/1139] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:42.395493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71105abb-4c15-43c3-962a-1da1de402f50 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:42.395939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71105abb-4c15-43c3-962a-1da1de402f50 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:42.396083 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71105abb-4c15-43c3-962a-1da1de402f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:42.412730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-71105abb-4c15-43c3-962a-1da1de402f50 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:42.413893 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-71105abb-4c15-43c3-962a-1da1de402f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:42.476189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 575/1140] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:42.499130 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c4923-21fe-4d12-93fb-a324ab260a44 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:42.595174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c4923-21fe-4d12-93fb-a324ab260a44 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:42.625553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063e17e-0d0f-45e2-bd12-52d2625454e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:42.626333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063e17e-0d0f-45e2-bd12-52d2625454e0 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:42.626609 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063e17e-0d0f-45e2-bd12-52d2625454e0 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:42.626874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063e17e-0d0f-45e2-bd12-52d2625454e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:42.636164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063e17e-0d0f-45e2-bd12-52d2625454e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-e063e17e-0d0f-45e2-bd12-52d2625454e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:14:42.644208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:42.645147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:42.646126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:42.646126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:42.646126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:42.646126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:42.646126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:14:42.646126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:14:42.646126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:14:42.646126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:14:42.646126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:14:42.646126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:14:42.646126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:14:42.646126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:42.646126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 566/1141] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:14:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:42.651322 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71105abb-4c15-43c3-962a-1da1de402f50 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:42.741826 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71105abb-4c15-43c3-962a-1da1de402f50 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:42.774139 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4381c-9ea5-4d2a-a78e-46d3c362ffa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:42.774795 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4381c-9ea5-4d2a-a78e-46d3c362ffa0 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:42.775072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4381c-9ea5-4d2a-a78e-46d3c362ffa0 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:42.775342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4381c-9ea5-4d2a-a78e-46d3c362ffa0 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:42.776110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4381c-9ea5-4d2a-a78e-46d3c362ffa0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:42.776420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4381c-9ea5-4d2a-a78e-46d3c362ffa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:42.777955 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4381c-9ea5-4d2a-a78e-46d3c362ffa0 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:42.785431 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 576/1142] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:14:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:42.791265 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063e17e-0d0f-45e2-bd12-52d2625454e0 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:42.891730 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063e17e-0d0f-45e2-bd12-52d2625454e0 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:42.924849 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c993fd-022c-4979-a4c0-0067d65e0086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:42.925636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c993fd-022c-4979-a4c0-0067d65e0086 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:42.925912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c993fd-022c-4979-a4c0-0067d65e0086 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:42.926181 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c993fd-022c-4979-a4c0-0067d65e0086 None admin] PATH_INFO: `/v3/users/44e5495b10e4460b96fa90a2c91e18e5` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:42.939593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c993fd-022c-4979-a4c0-0067d65e0086 None admin] RBAC: Authorizing `identity:get_user(user_id=44e5495b10e4460b96fa90a2c91e18e5)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:42.941046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c993fd-022c-4979-a4c0-0067d65e0086 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:42.942640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 567/1143] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:14:42 2022] GET /identity/v3/users/44e5495b10e4460b96fa90a2c91e18e5 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:42.948479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4381c-9ea5-4d2a-a78e-46d3c362ffa0 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:43.041784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4381c-9ea5-4d2a-a78e-46d3c362ffa0 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:43.072757 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616ba74-9c0e-4eba-85fa-f3c8887343de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:43.073383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616ba74-9c0e-4eba-85fa-f3c8887343de None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:43.073663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616ba74-9c0e-4eba-85fa-f3c8887343de None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:43.073935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616ba74-9c0e-4eba-85fa-f3c8887343de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:43.135670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616ba74-9c0e-4eba-85fa-f3c8887343de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:43.137154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616ba74-9c0e-4eba-85fa-f3c8887343de None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:43.232108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 577/1144] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:14:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:14:43.237998 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c993fd-022c-4979-a4c0-0067d65e0086 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:43.330729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c993fd-022c-4979-a4c0-0067d65e0086 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:43.362384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf4ccf-ae28-432f-adce-4958960deccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:43.363231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf4ccf-ae28-432f-adce-4958960deccf None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:43.363515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf4ccf-ae28-432f-adce-4958960deccf None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:43.363788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf4ccf-ae28-432f-adce-4958960deccf None admin] PATH_INFO: `/v3/projects/29f7d27adffa4a5089fc31116f09abe7` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:43.365002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf4ccf-ae28-432f-adce-4958960deccf None admin] RBAC: Authorizing `identity:get_project(project_id=29f7d27adffa4a5089fc31116f09abe7)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:43.366424 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf4ccf-ae28-432f-adce-4958960deccf None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:43.368111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 568/1145] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:14:43 2022] GET /identity/v3/projects/29f7d27adffa4a5089fc31116f09abe7 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:43.374883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616ba74-9c0e-4eba-85fa-f3c8887343de None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:43.465288 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616ba74-9c0e-4eba-85fa-f3c8887343de None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:43.496346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79da323-06d5-4746-87e3-97b6b76e90b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:43.496928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79da323-06d5-4746-87e3-97b6b76e90b5 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:43.497206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79da323-06d5-4746-87e3-97b6b76e90b5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:43.497474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79da323-06d5-4746-87e3-97b6b76e90b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:43.498287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79da323-06d5-4746-87e3-97b6b76e90b5 None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=29f7d27adffa4a5089fc31116f09abe7, user.id=44e5495b10e4460b96fa90a2c91e18e5) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:43.498592 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79da323-06d5-4746-87e3-97b6b76e90b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:43.500017 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79da323-06d5-4746-87e3-97b6b76e90b5 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:43.505058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 578/1146] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:14:43 2022] GET /identity/v3/role_assignments?user.id=44e5495b10e4460b96fa90a2c91e18e5&scope.project.id=29f7d27adffa4a5089fc31116f09abe7&role.id=e59113c0366049e889df3f55eed90c72 => generated 259 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:45.688789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e13923-eac4-43b7-a64f-47bce00e043a None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:45.689355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e13923-eac4-43b7-a64f-47bce00e043a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:45.689355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e13923-eac4-43b7-a64f-47bce00e043a None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:45.690671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 569/1147] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:45.695789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dd701-2cf5-46a6-a319-d1d4470980cd None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:45.696191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dd701-2cf5-46a6-a319-d1d4470980cd None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:45.696395 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dd701-2cf5-46a6-a319-d1d4470980cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:45.713441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-bb3dd701-2cf5-46a6-a319-d1d4470980cd None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:45.714602 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-bb3dd701-2cf5-46a6-a319-d1d4470980cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:45.777088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 579/1148] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:45.798817 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e13923-eac4-43b7-a64f-47bce00e043a None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:45.894145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e13923-eac4-43b7-a64f-47bce00e043a None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:45.924943 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3184810-9bb3-45cd-8449-00c6fa6a28eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:45.925719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3184810-9bb3-45cd-8449-00c6fa6a28eb None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:45.925997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3184810-9bb3-45cd-8449-00c6fa6a28eb None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:45.926269 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3184810-9bb3-45cd-8449-00c6fa6a28eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:45.936283 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3184810-9bb3-45cd-8449-00c6fa6a28eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-b3184810-9bb3-45cd-8449-00c6fa6a28eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:14:45.943693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:45.944818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:45.945990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:45.945990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:45.945990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:45.945990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:45.945990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:14:45.945990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:14:45.945990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:14:45.945990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:14:45.945990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:14:45.945990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:14:45.945990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:14:45.945990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:45.945990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 570/1149] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:14:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:45.951599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3dd701-2cf5-46a6-a319-d1d4470980cd None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:46.042378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3dd701-2cf5-46a6-a319-d1d4470980cd None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:46.073459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f181f1-5007-434e-9d0d-6819a128059e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:46.074682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f181f1-5007-434e-9d0d-6819a128059e None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:46.074965 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f181f1-5007-434e-9d0d-6819a128059e None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:46.075373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f181f1-5007-434e-9d0d-6819a128059e None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:46.076145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f181f1-5007-434e-9d0d-6819a128059e None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:46.076448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f181f1-5007-434e-9d0d-6819a128059e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:46.077812 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f181f1-5007-434e-9d0d-6819a128059e None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:46.085604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 580/1150] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:14:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:46.091438 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3184810-9bb3-45cd-8449-00c6fa6a28eb None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:46.192368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3184810-9bb3-45cd-8449-00c6fa6a28eb None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:46.223384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d697b98-806f-4f7a-a891-0b3ab8591f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:46.224271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d697b98-806f-4f7a-a891-0b3ab8591f89 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:46.224558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d697b98-806f-4f7a-a891-0b3ab8591f89 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:46.224835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d697b98-806f-4f7a-a891-0b3ab8591f89 None admin] PATH_INFO: `/v3/users/44e5495b10e4460b96fa90a2c91e18e5` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:46.226010 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d697b98-806f-4f7a-a891-0b3ab8591f89 None admin] RBAC: Authorizing `identity:get_user(user_id=44e5495b10e4460b96fa90a2c91e18e5)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:46.227437 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d697b98-806f-4f7a-a891-0b3ab8591f89 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:46.229102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 571/1151] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:14:46 2022] GET /identity/v3/users/44e5495b10e4460b96fa90a2c91e18e5 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:46.235558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f181f1-5007-434e-9d0d-6819a128059e None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:46.327225 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f181f1-5007-434e-9d0d-6819a128059e None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:46.358213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c478ca-f126-41bb-91e9-aa6d9966a116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:46.359054 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c478ca-f126-41bb-91e9-aa6d9966a116 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:46.359333 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c478ca-f126-41bb-91e9-aa6d9966a116 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:46.359606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c478ca-f126-41bb-91e9-aa6d9966a116 None admin] PATH_INFO: `/v3/projects/29f7d27adffa4a5089fc31116f09abe7` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:46.360794 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c478ca-f126-41bb-91e9-aa6d9966a116 None admin] RBAC: Authorizing `identity:get_project(project_id=29f7d27adffa4a5089fc31116f09abe7)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:46.362204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c478ca-f126-41bb-91e9-aa6d9966a116 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:46.363843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 581/1152] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:14:46 2022] GET /identity/v3/projects/29f7d27adffa4a5089fc31116f09abe7 => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:46.369545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d697b98-806f-4f7a-a891-0b3ab8591f89 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:46.461599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d697b98-806f-4f7a-a891-0b3ab8591f89 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:46.492467 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebba9e-c904-4fb0-b151-bf009b491940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:46.493273 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebba9e-c904-4fb0-b151-bf009b491940 None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:46.493549 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebba9e-c904-4fb0-b151-bf009b491940 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:46.493820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebba9e-c904-4fb0-b151-bf009b491940 None admin] PATH_INFO: `/v3/projects/29f7d27adffa4a5089fc31116f09abe7/users/44e5495b10e4460b96fa90a2c91e18e5/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:46.495728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebba9e-c904-4fb0-b151-bf009b491940 None admin] RBAC: Authorizing `identity:create_grant(project_id=29f7d27adffa4a5089fc31116f09abe7, user_id=44e5495b10e4460b96fa90a2c91e18e5, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:46.497678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebba9e-c904-4fb0-b151-bf009b491940 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:46.509796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 572/1153] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:14:46 2022] PUT /identity/v3/projects/29f7d27adffa4a5089fc31116f09abe7/users/44e5495b10e4460b96fa90a2c91e18e5/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:14:48.771711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835411fb-e225-4e08-915f-f01b0800cefb None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:48.772221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835411fb-e225-4e08-915f-f01b0800cefb None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:48.772315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835411fb-e225-4e08-915f-f01b0800cefb None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:48.773668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 582/1154] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:48.778796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc4ad4-897b-42f5-8085-e38dd08d5430 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:48.779222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc4ad4-897b-42f5-8085-e38dd08d5430 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:48.779356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc4ad4-897b-42f5-8085-e38dd08d5430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:48.795539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-43fc4ad4-897b-42f5-8085-e38dd08d5430 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:48.796739 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-43fc4ad4-897b-42f5-8085-e38dd08d5430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:48.858986 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 573/1155] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:48.882320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835411fb-e225-4e08-915f-f01b0800cefb None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:48.976802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835411fb-e225-4e08-915f-f01b0800cefb None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:49.007829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefc17c-93bf-491c-93f6-cc48f06473a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:49.008630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefc17c-93bf-491c-93f6-cc48f06473a8 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:49.008962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefc17c-93bf-491c-93f6-cc48f06473a8 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:49.009180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefc17c-93bf-491c-93f6-cc48f06473a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:49.020077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aefc17c-93bf-491c-93f6-cc48f06473a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-1aefc17c-93bf-491c-93f6-cc48f06473a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:14:49.028219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:49.029329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:49.030166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:49.030166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:49.030166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:49.030166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:49.030166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:14:49.030166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:14:49.030166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:14:49.030166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:14:49.030166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:14:49.030166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:14:49.030166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:14:49.030166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:14:49.030166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 583/1156] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:49.035169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc4ad4-897b-42f5-8085-e38dd08d5430 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:49.134454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc4ad4-897b-42f5-8085-e38dd08d5430 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:49.166622 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1ab74-1bdd-4038-905d-cd22c0ff8bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:49.167265 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1ab74-1bdd-4038-905d-cd22c0ff8bd9 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:49.167545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1ab74-1bdd-4038-905d-cd22c0ff8bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:49.167875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1ab74-1bdd-4038-905d-cd22c0ff8bd9 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:49.168640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1ab74-1bdd-4038-905d-cd22c0ff8bd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:49.168996 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1ab74-1bdd-4038-905d-cd22c0ff8bd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:49.170340 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1ab74-1bdd-4038-905d-cd22c0ff8bd9 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:49.177809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 574/1157] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:14:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:49.183847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefc17c-93bf-491c-93f6-cc48f06473a8 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:49.290960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefc17c-93bf-491c-93f6-cc48f06473a8 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:49.322581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cb4c7-5f1e-4f73-b27f-71ff9f6f4e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:49.323430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cb4c7-5f1e-4f73-b27f-71ff9f6f4e97 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:49.323708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cb4c7-5f1e-4f73-b27f-71ff9f6f4e97 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:49.324045 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cb4c7-5f1e-4f73-b27f-71ff9f6f4e97 None admin] PATH_INFO: `/v3/users/44e5495b10e4460b96fa90a2c91e18e5` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:49.325369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cb4c7-5f1e-4f73-b27f-71ff9f6f4e97 None admin] RBAC: Authorizing `identity:get_user(user_id=44e5495b10e4460b96fa90a2c91e18e5)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:49.326781 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cb4c7-5f1e-4f73-b27f-71ff9f6f4e97 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:49.328408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 584/1158] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:14:49 2022] GET /identity/v3/users/44e5495b10e4460b96fa90a2c91e18e5 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:49.334372 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1ab74-1bdd-4038-905d-cd22c0ff8bd9 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:49.428028 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1ab74-1bdd-4038-905d-cd22c0ff8bd9 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:49.459711 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13295a-68c1-444f-98ac-3cdbac489afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:49.460383 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13295a-68c1-444f-98ac-3cdbac489afc None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:49.460672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13295a-68c1-444f-98ac-3cdbac489afc None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:49.460951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13295a-68c1-444f-98ac-3cdbac489afc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:49.524135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13295a-68c1-444f-98ac-3cdbac489afc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:49.525603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13295a-68c1-444f-98ac-3cdbac489afc None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:49.618083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 575/1159] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:14:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:14:49.623920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cb4c7-5f1e-4f73-b27f-71ff9f6f4e97 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:49.714106 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cb4c7-5f1e-4f73-b27f-71ff9f6f4e97 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:49.745998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb612bfd-7725-45d5-b8e2-780ec540e136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:49.746763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb612bfd-7725-45d5-b8e2-780ec540e136 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:49.747045 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb612bfd-7725-45d5-b8e2-780ec540e136 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:49.747313 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb612bfd-7725-45d5-b8e2-780ec540e136 None admin] PATH_INFO: `/v3/projects/29f7d27adffa4a5089fc31116f09abe7` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:49.748499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb612bfd-7725-45d5-b8e2-780ec540e136 None admin] RBAC: Authorizing `identity:get_project(project_id=29f7d27adffa4a5089fc31116f09abe7)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:49.749919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb612bfd-7725-45d5-b8e2-780ec540e136 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:49.751579 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 585/1160] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:14:49 2022] GET /identity/v3/projects/29f7d27adffa4a5089fc31116f09abe7 => generated 311 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:49.757720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13295a-68c1-444f-98ac-3cdbac489afc None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:49.850254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13295a-68c1-444f-98ac-3cdbac489afc None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:49.880969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312691d-0ad5-4678-955d-5beebf559c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:49.881550 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312691d-0ad5-4678-955d-5beebf559c0a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:49.881826 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312691d-0ad5-4678-955d-5beebf559c0a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:49.882098 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312691d-0ad5-4678-955d-5beebf559c0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:49.882907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312691d-0ad5-4678-955d-5beebf559c0a None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=29f7d27adffa4a5089fc31116f09abe7, user.id=44e5495b10e4460b96fa90a2c91e18e5) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:49.883213 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312691d-0ad5-4678-955d-5beebf559c0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:49.884645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312691d-0ad5-4678-955d-5beebf559c0a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:49.890605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 576/1161] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:14:49 2022] GET /identity/v3/role_assignments?user.id=44e5495b10e4460b96fa90a2c91e18e5&scope.project.id=29f7d27adffa4a5089fc31116f09abe7&role.id=e59113c0366049e889df3f55eed90c72 => generated 608 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:52.141696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4041806-cf07-48a7-8ee6-7c3c2a746bf3 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:52.142371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4041806-cf07-48a7-8ee6-7c3c2a746bf3 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:52.142371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4041806-cf07-48a7-8ee6-7c3c2a746bf3 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:52.144388 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 586/1162] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:52.155288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d5d3c-8beb-48ec-b313-43e82a9a1076 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:52.155735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d5d3c-8beb-48ec-b313-43e82a9a1076 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:52.155836 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d5d3c-8beb-48ec-b313-43e82a9a1076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:52.171851 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-898d5d3c-8beb-48ec-b313-43e82a9a1076 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:52.173035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-898d5d3c-8beb-48ec-b313-43e82a9a1076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:52.238615 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 577/1163] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:52.260452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4041806-cf07-48a7-8ee6-7c3c2a746bf3 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:52.355862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4041806-cf07-48a7-8ee6-7c3c2a746bf3 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:52.387558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05078976-1120-4b8f-8337-5691307e7107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:52.388210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05078976-1120-4b8f-8337-5691307e7107 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:52.388494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05078976-1120-4b8f-8337-5691307e7107 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:52.388768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05078976-1120-4b8f-8337-5691307e7107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:52.450800 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05078976-1120-4b8f-8337-5691307e7107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:52.452188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05078976-1120-4b8f-8337-5691307e7107 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:52.550219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 587/1164] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:14:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:14:52.556217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d5d3c-8beb-48ec-b313-43e82a9a1076 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:52.648365 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d5d3c-8beb-48ec-b313-43e82a9a1076 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:52.679458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd57a9-77db-4884-af3a-96c0035ee6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:52.680272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd57a9-77db-4884-af3a-96c0035ee6d4 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:52.680555 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd57a9-77db-4884-af3a-96c0035ee6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:52.680826 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd57a9-77db-4884-af3a-96c0035ee6d4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:52.687542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd57a9-77db-4884-af3a-96c0035ee6d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:52.688995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd57a9-77db-4884-af3a-96c0035ee6d4 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-49cd57a9-77db-4884-af3a-96c0035ee6d4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:14:52.702995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:14:52.704554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:14:52.705475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 578/1165] 10.222.0.44 () {56 vars in 1117 bytes} [Mon May 9 04:14:52 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:14:52.710142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05078976-1120-4b8f-8337-5691307e7107 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:52.800267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05078976-1120-4b8f-8337-5691307e7107 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:52.831983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f40dc-60f6-46d6-83de-d10f460a2ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:52.832628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f40dc-60f6-46d6-83de-d10f460a2ca1 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:52.832906 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f40dc-60f6-46d6-83de-d10f460a2ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:52.833176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f40dc-60f6-46d6-83de-d10f460a2ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:52.833884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f40dc-60f6-46d6-83de-d10f460a2ca1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:14:52.834182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f40dc-60f6-46d6-83de-d10f460a2ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:52.835529 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f40dc-60f6-46d6-83de-d10f460a2ca1 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:52.842226 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 588/1166] 10.222.0.44 () {56 vars in 1115 bytes} [Mon May 9 04:14:52 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:55.028509 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83d01e-ae74-484c-991c-3f4b1f2964ab None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:55.029023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83d01e-ae74-484c-991c-3f4b1f2964ab None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:55.029023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83d01e-ae74-484c-991c-3f4b1f2964ab None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:55.030344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 579/1167] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:55.035743 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed576f-63dc-4bde-82ba-e53397d231f3 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:55.036170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed576f-63dc-4bde-82ba-e53397d231f3 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:55.036325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed576f-63dc-4bde-82ba-e53397d231f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:55.054139 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-cbed576f-63dc-4bde-82ba-e53397d231f3 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:55.055292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cbed576f-63dc-4bde-82ba-e53397d231f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:55.117299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 589/1168] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:55.140646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83d01e-ae74-484c-991c-3f4b1f2964ab None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:55.237450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83d01e-ae74-484c-991c-3f4b1f2964ab None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:55.268558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0e0e3-d405-488b-b45d-06594ae02044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:55.269200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0e0e3-d405-488b-b45d-06594ae02044 None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:55.269486 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0e0e3-d405-488b-b45d-06594ae02044 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:55.269765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0e0e3-d405-488b-b45d-06594ae02044 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:55.270494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0e0e3-d405-488b-b45d-06594ae02044 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:55.271848 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0e0e3-d405-488b-b45d-06594ae02044 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:55.285992 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 580/1169] 10.222.0.44 () {60 vars in 1138 bytes} [Mon May 9 04:14:55 2022] POST /identity/v3/domains => generated 267 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:14:57.514059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e139c77-62a2-493b-a41d-bf5c28302eb6 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:57.514613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e139c77-62a2-493b-a41d-bf5c28302eb6 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:57.514613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e139c77-62a2-493b-a41d-bf5c28302eb6 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:57.515986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 590/1170] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:14:57.521251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e886b-bcfd-4353-acd9-c8ed773c85f3 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:57.521676 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e886b-bcfd-4353-acd9-c8ed773c85f3 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:57.521810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e886b-bcfd-4353-acd9-c8ed773c85f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:57.538675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-477e886b-bcfd-4353-acd9-c8ed773c85f3 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:14:57.539829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-477e886b-bcfd-4353-acd9-c8ed773c85f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:14:57.617711 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 581/1171] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:14:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:14:57.641301 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e139c77-62a2-493b-a41d-bf5c28302eb6 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:57.734334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e139c77-62a2-493b-a41d-bf5c28302eb6 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:57.766209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b4e0a-00c9-43b3-8d63-34ff7c5eed2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:57.767007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b4e0a-00c9-43b3-8d63-34ff7c5eed2d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:57.767285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b4e0a-00c9-43b3-8d63-34ff7c5eed2d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:57.767554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b4e0a-00c9-43b3-8d63-34ff7c5eed2d None admin] PATH_INFO: `/v3/domains/9ab186163c1a44dea2545c31d30d066b` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:57.777496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b4e0a-00c9-43b3-8d63-34ff7c5eed2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=9ab186163c1a44dea2545c31d30d066b)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:57.778923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b4e0a-00c9-43b3-8d63-34ff7c5eed2d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:57.780564 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 591/1172] 10.222.0.44 () {56 vars in 1183 bytes} [Mon May 9 04:14:57 2022] GET /identity/v3/domains/9ab186163c1a44dea2545c31d30d066b => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:14:57.786353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e886b-bcfd-4353-acd9-c8ed773c85f3 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:14:57.877720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e886b-bcfd-4353-acd9-c8ed773c85f3 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:14:57.909251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0685d9b-8d4f-4f98-a086-d82560d6e60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:14:57.909892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0685d9b-8d4f-4f98-a086-d82560d6e60f None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:14:57.910173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0685d9b-8d4f-4f98-a086-d82560d6e60f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:14:57.910448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0685d9b-8d4f-4f98-a086-d82560d6e60f None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:14:57.911145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0685d9b-8d4f-4f98-a086-d82560d6e60f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:14:57.912582 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0685d9b-8d4f-4f98-a086-d82560d6e60f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:14:57.937401 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 582/1173] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:14:57 2022] POST /identity/v3/projects => generated 361 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:15:00.290732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acff39-8b91-4c33-bae6-60aa24b36c01 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:00.291256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acff39-8b91-4c33-bae6-60aa24b36c01 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:00.291256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acff39-8b91-4c33-bae6-60aa24b36c01 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:00.293224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 592/1174] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:15:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:15:00.298616 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5a063-f5f7-498a-9d7c-e7ccd05547a6 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:00.299041 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5a063-f5f7-498a-9d7c-e7ccd05547a6 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:00.299167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5a063-f5f7-498a-9d7c-e7ccd05547a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:00.316100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-10e5a063-f5f7-498a-9d7c-e7ccd05547a6 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:15:00.317455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-10e5a063-f5f7-498a-9d7c-e7ccd05547a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:15:00.379856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 583/1175] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:15:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:15:00.404565 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14acff39-8b91-4c33-bae6-60aa24b36c01 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:00.497711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14acff39-8b91-4c33-bae6-60aa24b36c01 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:00.528150 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d720b58-4dbf-4e10-bb34-4916d279426b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:00.529004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d720b58-4dbf-4e10-bb34-4916d279426b None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:00.529283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d720b58-4dbf-4e10-bb34-4916d279426b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:00.529552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d720b58-4dbf-4e10-bb34-4916d279426b None admin] PATH_INFO: `/v3/domains/9ab186163c1a44dea2545c31d30d066b` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:00.530703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d720b58-4dbf-4e10-bb34-4916d279426b None admin] RBAC: Authorizing `identity:get_domain(domain_id=9ab186163c1a44dea2545c31d30d066b)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:00.532129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d720b58-4dbf-4e10-bb34-4916d279426b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:00.533705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 593/1176] 10.222.0.44 () {56 vars in 1183 bytes} [Mon May 9 04:15:00 2022] GET /identity/v3/domains/9ab186163c1a44dea2545c31d30d066b => generated 267 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:15:00.539366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5a063-f5f7-498a-9d7c-e7ccd05547a6 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:00.631535 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5a063-f5f7-498a-9d7c-e7ccd05547a6 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:00.662236 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefb7ad-34d7-4213-a0ed-3457b466eb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:00.662889 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefb7ad-34d7-4213-a0ed-3457b466eb69 None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:00.663172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefb7ad-34d7-4213-a0ed-3457b466eb69 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:00.663447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefb7ad-34d7-4213-a0ed-3457b466eb69 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:00.664185 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefb7ad-34d7-4213-a0ed-3457b466eb69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:00.665593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefb7ad-34d7-4213-a0ed-3457b466eb69 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:00.684203 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 584/1177] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:15:00 2022] POST /identity/v3/users => generated 311 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:15:02.943992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85a36f-4378-42a0-9384-ad759c874ea8 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:02.944533 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85a36f-4378-42a0-9384-ad759c874ea8 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:02.944533 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85a36f-4378-42a0-9384-ad759c874ea8 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:02.945915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 594/1178] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:15:02.951539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895133ea-f1e8-4e8f-b021-7ad7c5a0a140 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:02.951984 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895133ea-f1e8-4e8f-b021-7ad7c5a0a140 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:02.952218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895133ea-f1e8-4e8f-b021-7ad7c5a0a140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:02.969020 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-895133ea-f1e8-4e8f-b021-7ad7c5a0a140 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:15:02.970177 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-895133ea-f1e8-4e8f-b021-7ad7c5a0a140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:15:03.031785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 585/1179] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:15:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:15:03.054415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85a36f-4378-42a0-9384-ad759c874ea8 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:03.149645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85a36f-4378-42a0-9384-ad759c874ea8 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:03.179840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3641cb-2903-4482-ab22-e7b8c89b8993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:03.180640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3641cb-2903-4482-ab22-e7b8c89b8993 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:03.180921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3641cb-2903-4482-ab22-e7b8c89b8993 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:03.181194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3641cb-2903-4482-ab22-e7b8c89b8993 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:03.191483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3641cb-2903-4482-ab22-e7b8c89b8993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-dd3641cb-2903-4482-ab22-e7b8c89b8993 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:15:03.199671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:15:03.200815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:15:03.201855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:15:03.201855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:15:03.201855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:15:03.201855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:15:03.201855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:15:03.201855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:15:03.201855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:15:03.201855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:15:03.201855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:15:03.201855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:15:03.201855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:15:03.201855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:15:03.201855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 595/1180] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:15:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:15:03.208825 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895133ea-f1e8-4e8f-b021-7ad7c5a0a140 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:03.298419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895133ea-f1e8-4e8f-b021-7ad7c5a0a140 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:03.330139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a90fc-ca23-47ea-99ae-fe471824691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:03.330782 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a90fc-ca23-47ea-99ae-fe471824691b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:03.331060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a90fc-ca23-47ea-99ae-fe471824691b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:03.331330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a90fc-ca23-47ea-99ae-fe471824691b None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:03.332085 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a90fc-ca23-47ea-99ae-fe471824691b None admin] RBAC: Adding query filter params (name=admin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:15:03.332386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a90fc-ca23-47ea-99ae-fe471824691b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:03.333721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a90fc-ca23-47ea-99ae-fe471824691b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:03.341253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 586/1181] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:15:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:15:03.346974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3641cb-2903-4482-ab22-e7b8c89b8993 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:03.438649 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3641cb-2903-4482-ab22-e7b8c89b8993 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:03.469498 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40565e-1fca-42d7-86ac-ec9035fe30ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:03.470351 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40565e-1fca-42d7-86ac-ec9035fe30ce None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:03.470630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40565e-1fca-42d7-86ac-ec9035fe30ce None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:03.470898 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40565e-1fca-42d7-86ac-ec9035fe30ce None admin] PATH_INFO: `/v3/users/6ada441ab15d4ca894188f027ddb0630` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:03.483861 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40565e-1fca-42d7-86ac-ec9035fe30ce None admin] RBAC: Authorizing `identity:get_user(user_id=6ada441ab15d4ca894188f027ddb0630)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:03.485306 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40565e-1fca-42d7-86ac-ec9035fe30ce None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:03.486900 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 596/1182] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:15:03 2022] GET /identity/v3/users/6ada441ab15d4ca894188f027ddb0630 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:15:03.492523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a90fc-ca23-47ea-99ae-fe471824691b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:03.586656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a90fc-ca23-47ea-99ae-fe471824691b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:03.618480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2f0bc-67f4-4f3e-a354-b2b594d28cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:03.619118 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2f0bc-67f4-4f3e-a354-b2b594d28cd9 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:03.619404 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2f0bc-67f4-4f3e-a354-b2b594d28cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:03.619696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2f0bc-67f4-4f3e-a354-b2b594d28cd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:03.684319 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2f0bc-67f4-4f3e-a354-b2b594d28cd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:03.685743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2f0bc-67f4-4f3e-a354-b2b594d28cd9 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:03.778443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 587/1183] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:15:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:15:03.784249 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40565e-1fca-42d7-86ac-ec9035fe30ce None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:03.877430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40565e-1fca-42d7-86ac-ec9035fe30ce None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:03.908406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b92b7-1488-4e78-a923-226520d2fc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:03.909244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b92b7-1488-4e78-a923-226520d2fc4c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:03.909524 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b92b7-1488-4e78-a923-226520d2fc4c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:03.909799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b92b7-1488-4e78-a923-226520d2fc4c None admin] PATH_INFO: `/v3/projects/862ce5cf5ac44e57ba1d13a2dc622383` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:03.910950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b92b7-1488-4e78-a923-226520d2fc4c None admin] RBAC: Authorizing `identity:get_project(project_id=862ce5cf5ac44e57ba1d13a2dc622383)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:03.912403 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b92b7-1488-4e78-a923-226520d2fc4c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:03.914051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 597/1184] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:15:03 2022] GET /identity/v3/projects/862ce5cf5ac44e57ba1d13a2dc622383 => generated 361 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:15:03.920747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2f0bc-67f4-4f3e-a354-b2b594d28cd9 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:04.012278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2f0bc-67f4-4f3e-a354-b2b594d28cd9 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:04.044124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf672b-5146-4f2f-a0b0-8027107a0abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:04.044762 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf672b-5146-4f2f-a0b0-8027107a0abc None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:04.045047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf672b-5146-4f2f-a0b0-8027107a0abc None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:04.045325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf672b-5146-4f2f-a0b0-8027107a0abc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:04.046148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf672b-5146-4f2f-a0b0-8027107a0abc None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=862ce5cf5ac44e57ba1d13a2dc622383, user.id=6ada441ab15d4ca894188f027ddb0630) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:15:04.046461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf672b-5146-4f2f-a0b0-8027107a0abc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:04.047867 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf672b-5146-4f2f-a0b0-8027107a0abc None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:04.053005 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 588/1185] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:15:03 2022] GET /identity/v3/role_assignments?user.id=6ada441ab15d4ca894188f027ddb0630&scope.project.id=862ce5cf5ac44e57ba1d13a2dc622383&role.id=e59113c0366049e889df3f55eed90c72 => generated 259 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:15:06.229065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d082037-31a2-4b45-bf3c-89ff4a16d596 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:06.229590 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d082037-31a2-4b45-bf3c-89ff4a16d596 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:06.229590 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d082037-31a2-4b45-bf3c-89ff4a16d596 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:06.231704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 598/1186] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:15:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:15:06.237048 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b223e-13b5-43a2-aeca-fa5c4ed32b74 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:06.237468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b223e-13b5-43a2-aeca-fa5c4ed32b74 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:06.237610 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b223e-13b5-43a2-aeca-fa5c4ed32b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:06.254096 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d11b223e-13b5-43a2-aeca-fa5c4ed32b74 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:15:06.255258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d11b223e-13b5-43a2-aeca-fa5c4ed32b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:15:06.316618 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 589/1187] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:15:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:15:06.339118 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d082037-31a2-4b45-bf3c-89ff4a16d596 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:06.433917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d082037-31a2-4b45-bf3c-89ff4a16d596 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:06.464996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bf1ae-3e16-4ad5-80c7-0bfce157c1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:06.465767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bf1ae-3e16-4ad5-80c7-0bfce157c1a6 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:06.466050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bf1ae-3e16-4ad5-80c7-0bfce157c1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:06.466324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bf1ae-3e16-4ad5-80c7-0bfce157c1a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:06.475644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262bf1ae-3e16-4ad5-80c7-0bfce157c1a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-262bf1ae-3e16-4ad5-80c7-0bfce157c1a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:15:06.483810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:15:06.484941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:15:06.485971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:15:06.485971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:15:06.485971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:15:06.485971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:15:06.485971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:15:06.485971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:15:06.485971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:15:06.485971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:15:06.485971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:15:06.485971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:15:06.485971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:15:06.485971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:15:06.485971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 599/1188] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:15:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:15:06.490758 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b223e-13b5-43a2-aeca-fa5c4ed32b74 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:06.583468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b223e-13b5-43a2-aeca-fa5c4ed32b74 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:06.615582 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469b79e-d858-4e80-8e7c-284ea2df9f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:06.616272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469b79e-d858-4e80-8e7c-284ea2df9f0a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:06.616553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469b79e-d858-4e80-8e7c-284ea2df9f0a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:06.616826 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469b79e-d858-4e80-8e7c-284ea2df9f0a None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:06.617557 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469b79e-d858-4e80-8e7c-284ea2df9f0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:15:06.617856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469b79e-d858-4e80-8e7c-284ea2df9f0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:06.619204 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469b79e-d858-4e80-8e7c-284ea2df9f0a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:06.626895 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 590/1189] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:15:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:15:06.632766 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bf1ae-3e16-4ad5-80c7-0bfce157c1a6 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:06.724524 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bf1ae-3e16-4ad5-80c7-0bfce157c1a6 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:06.755438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2fc4b-4a7b-4f6b-a97f-a6171eca3c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:06.756272 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2fc4b-4a7b-4f6b-a97f-a6171eca3c4c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:06.756554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2fc4b-4a7b-4f6b-a97f-a6171eca3c4c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:06.756825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2fc4b-4a7b-4f6b-a97f-a6171eca3c4c None admin] PATH_INFO: `/v3/users/6ada441ab15d4ca894188f027ddb0630` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:06.758497 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2fc4b-4a7b-4f6b-a97f-a6171eca3c4c None admin] RBAC: Authorizing `identity:get_user(user_id=6ada441ab15d4ca894188f027ddb0630)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:06.759383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2fc4b-4a7b-4f6b-a97f-a6171eca3c4c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:06.761012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 600/1190] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:15:06 2022] GET /identity/v3/users/6ada441ab15d4ca894188f027ddb0630 => generated 311 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:15:06.767199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469b79e-d858-4e80-8e7c-284ea2df9f0a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:06.858522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469b79e-d858-4e80-8e7c-284ea2df9f0a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:06.890089 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3982c26-f25e-4761-8f8c-1567ec657c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:06.890910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3982c26-f25e-4761-8f8c-1567ec657c55 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:06.891189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3982c26-f25e-4761-8f8c-1567ec657c55 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:06.891459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3982c26-f25e-4761-8f8c-1567ec657c55 None admin] PATH_INFO: `/v3/projects/862ce5cf5ac44e57ba1d13a2dc622383` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:06.892617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3982c26-f25e-4761-8f8c-1567ec657c55 None admin] RBAC: Authorizing `identity:get_project(project_id=862ce5cf5ac44e57ba1d13a2dc622383)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:06.894016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3982c26-f25e-4761-8f8c-1567ec657c55 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:06.895645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 591/1191] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:15:06 2022] GET /identity/v3/projects/862ce5cf5ac44e57ba1d13a2dc622383 => generated 361 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:15:06.901385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2fc4b-4a7b-4f6b-a97f-a6171eca3c4c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:06.994676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2fc4b-4a7b-4f6b-a97f-a6171eca3c4c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:07.026652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12582e6-eabc-4f48-887d-6088227fac42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:07.028032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12582e6-eabc-4f48-887d-6088227fac42 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:07.028032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12582e6-eabc-4f48-887d-6088227fac42 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:07.028228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12582e6-eabc-4f48-887d-6088227fac42 None admin] PATH_INFO: `/v3/projects/862ce5cf5ac44e57ba1d13a2dc622383/users/6ada441ab15d4ca894188f027ddb0630/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:07.030150 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12582e6-eabc-4f48-887d-6088227fac42 None admin] RBAC: Authorizing `identity:create_grant(project_id=862ce5cf5ac44e57ba1d13a2dc622383, user_id=6ada441ab15d4ca894188f027ddb0630, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:07.031578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12582e6-eabc-4f48-887d-6088227fac42 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:07.043999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 601/1192] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:15:06 2022] PUT /identity/v3/projects/862ce5cf5ac44e57ba1d13a2dc622383/users/6ada441ab15d4ca894188f027ddb0630/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:15:09.236420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9e091-60bd-4392-8278-8c3d620db986 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:09.236934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9e091-60bd-4392-8278-8c3d620db986 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:09.237042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9e091-60bd-4392-8278-8c3d620db986 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:09.238334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 592/1193] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:15:09.243473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73f670-4557-474f-bba9-28921a42a8d8 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:09.243924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73f670-4557-474f-bba9-28921a42a8d8 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:09.244067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73f670-4557-474f-bba9-28921a42a8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:09.260660 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-4e73f670-4557-474f-bba9-28921a42a8d8 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:15:09.262491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4e73f670-4557-474f-bba9-28921a42a8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:15:09.323200 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 602/1194] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:15:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 04:15:09.347159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9e091-60bd-4392-8278-8c3d620db986 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:09.442518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9e091-60bd-4392-8278-8c3d620db986 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:09.472756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650f12c-fd95-4b52-b17a-d0eaf0a0894c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:09.473526 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650f12c-fd95-4b52-b17a-d0eaf0a0894c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:09.473803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650f12c-fd95-4b52-b17a-d0eaf0a0894c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:09.474069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650f12c-fd95-4b52-b17a-d0eaf0a0894c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:09.484301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650f12c-fd95-4b52-b17a-d0eaf0a0894c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-7650f12c-fd95-4b52-b17a-d0eaf0a0894c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:15:09.491726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:15:09.492683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:15:09.493515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:15:09.493515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:15:09.493515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:15:09.493515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:15:09.493515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:15:09.493515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:15:09.493515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:15:09.493515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:15:09.493515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:15:09.493515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:15:09.493515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:15:09.493515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:15:09.493515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 593/1195] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:15:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:15:09.499182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73f670-4557-474f-bba9-28921a42a8d8 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:09.589542 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73f670-4557-474f-bba9-28921a42a8d8 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:09.620267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640596cf-17a3-43a6-b6e8-e914ac5bbb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:09.621454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640596cf-17a3-43a6-b6e8-e914ac5bbb14 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:09.621728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640596cf-17a3-43a6-b6e8-e914ac5bbb14 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:09.621999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640596cf-17a3-43a6-b6e8-e914ac5bbb14 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:09.622755 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640596cf-17a3-43a6-b6e8-e914ac5bbb14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:15:09.623053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640596cf-17a3-43a6-b6e8-e914ac5bbb14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:09.624428 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640596cf-17a3-43a6-b6e8-e914ac5bbb14 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:09.631871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 603/1196] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:15:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:15:09.637874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650f12c-fd95-4b52-b17a-d0eaf0a0894c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:09.729373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650f12c-fd95-4b52-b17a-d0eaf0a0894c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:09.761845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe2e83-086c-42b5-b60c-487cac9506bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:09.762707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe2e83-086c-42b5-b60c-487cac9506bb None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:09.762980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe2e83-086c-42b5-b60c-487cac9506bb None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:09.763248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe2e83-086c-42b5-b60c-487cac9506bb None admin] PATH_INFO: `/v3/users/6ada441ab15d4ca894188f027ddb0630` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:09.764443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe2e83-086c-42b5-b60c-487cac9506bb None admin] RBAC: Authorizing `identity:get_user(user_id=6ada441ab15d4ca894188f027ddb0630)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:09.765856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe2e83-086c-42b5-b60c-487cac9506bb None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:09.767451 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 594/1197] 10.222.0.44 () {56 vars in 1177 bytes} [Mon May 9 04:15:09 2022] GET /identity/v3/users/6ada441ab15d4ca894188f027ddb0630 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:15:09.773123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640596cf-17a3-43a6-b6e8-e914ac5bbb14 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:09.864925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640596cf-17a3-43a6-b6e8-e914ac5bbb14 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:09.895350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4431bb7-c66f-485a-a3d7-33da2d01cca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:09.896009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4431bb7-c66f-485a-a3d7-33da2d01cca8 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:09.896290 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4431bb7-c66f-485a-a3d7-33da2d01cca8 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:09.896560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4431bb7-c66f-485a-a3d7-33da2d01cca8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:09.960134 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4431bb7-c66f-485a-a3d7-33da2d01cca8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:09.961557 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4431bb7-c66f-485a-a3d7-33da2d01cca8 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:10.050804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 604/1198] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:15:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:15:10.056548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe2e83-086c-42b5-b60c-487cac9506bb None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:10.147530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe2e83-086c-42b5-b60c-487cac9506bb None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:10.178772 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b327b-69c1-4f47-9dad-94811547b59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:10.179605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b327b-69c1-4f47-9dad-94811547b59e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:10.179886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b327b-69c1-4f47-9dad-94811547b59e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:10.180186 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b327b-69c1-4f47-9dad-94811547b59e None admin] PATH_INFO: `/v3/projects/862ce5cf5ac44e57ba1d13a2dc622383` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:10.181321 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b327b-69c1-4f47-9dad-94811547b59e None admin] RBAC: Authorizing `identity:get_project(project_id=862ce5cf5ac44e57ba1d13a2dc622383)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:10.182723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b327b-69c1-4f47-9dad-94811547b59e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:10.184418 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 595/1199] 10.222.0.44 () {56 vars in 1186 bytes} [Mon May 9 04:15:10 2022] GET /identity/v3/projects/862ce5cf5ac44e57ba1d13a2dc622383 => generated 361 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:15:10.190255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4431bb7-c66f-485a-a3d7-33da2d01cca8 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:15:10.282678 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4431bb7-c66f-485a-a3d7-33da2d01cca8 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:15:10.313697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549337e-0bf9-444b-b381-50f0202a4fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:15:10.314276 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549337e-0bf9-444b-b381-50f0202a4fc3 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:15:10.314559 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549337e-0bf9-444b-b381-50f0202a4fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:15:10.314836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549337e-0bf9-444b-b381-50f0202a4fc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:15:10.315660 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549337e-0bf9-444b-b381-50f0202a4fc3 None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=862ce5cf5ac44e57ba1d13a2dc622383, user.id=6ada441ab15d4ca894188f027ddb0630) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:15:10.315995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549337e-0bf9-444b-b381-50f0202a4fc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:15:10.317408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549337e-0bf9-444b-b381-50f0202a4fc3 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:15:10.322860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 605/1200] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:15:10 2022] GET /identity/v3/role_assignments?user.id=6ada441ab15d4ca894188f027ddb0630&scope.project.id=862ce5cf5ac44e57ba1d13a2dc622383&role.id=e59113c0366049e889df3f55eed90c72 => generated 608 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:11.132026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a125808-5e99-459c-94af-c3abe9176809 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:11.132587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a125808-5e99-459c-94af-c3abe9176809 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:11.132587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a125808-5e99-459c-94af-c3abe9176809 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:11.134124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 596/1201] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:16:11.139705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2fa2b-2dfe-4159-a42e-c2d6b0da085c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:11.140284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2fa2b-2dfe-4159-a42e-c2d6b0da085c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:11.140284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2fa2b-2dfe-4159-a42e-c2d6b0da085c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:11.159838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-64a2fa2b-2dfe-4159-a42e-c2d6b0da085c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:16:11.161120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-64a2fa2b-2dfe-4159-a42e-c2d6b0da085c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:16:11.260092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 606/1202] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:16:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:16:11.273173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a125808-5e99-459c-94af-c3abe9176809 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:11.401706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a125808-5e99-459c-94af-c3abe9176809 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:11.439504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384be26-1519-4db4-bb5a-197d64ca915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:11.440348 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384be26-1519-4db4-bb5a-197d64ca915e None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:11.440629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384be26-1519-4db4-bb5a-197d64ca915e None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:11.440929 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384be26-1519-4db4-bb5a-197d64ca915e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:11.453601 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384be26-1519-4db4-bb5a-197d64ca915e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:11.455580 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384be26-1519-4db4-bb5a-197d64ca915e None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-a384be26-1519-4db4-bb5a-197d64ca915e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:16:11.480287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:11.481389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:11.482368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:16:11.482368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:16:11.482368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:11.482368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:11.482368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 597/1203] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:11.490152 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2fa2b-2dfe-4159-a42e-c2d6b0da085c None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:11.602889 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2fa2b-2dfe-4159-a42e-c2d6b0da085c None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:11.644766 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea4d58-bb78-457c-abc9-9a21ae48e7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:11.645542 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea4d58-bb78-457c-abc9-9a21ae48e7dd None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:11.645885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea4d58-bb78-457c-abc9-9a21ae48e7dd None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:11.646207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea4d58-bb78-457c-abc9-9a21ae48e7dd None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:11.646965 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea4d58-bb78-457c-abc9-9a21ae48e7dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:11.647299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea4d58-bb78-457c-abc9-9a21ae48e7dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:11.648901 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea4d58-bb78-457c-abc9-9a21ae48e7dd None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:11.662536 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 607/1204] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:11.668299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384be26-1519-4db4-bb5a-197d64ca915e None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:11.811828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384be26-1519-4db4-bb5a-197d64ca915e None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:11.866599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac176d-a7e8-4bb4-8cac-337593e7ab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:11.866599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac176d-a7e8-4bb4-8cac-337593e7ab93 None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:11.866599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac176d-a7e8-4bb4-8cac-337593e7ab93 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:11.866599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac176d-a7e8-4bb4-8cac-337593e7ab93 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:11.867086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac176d-a7e8-4bb4-8cac-337593e7ab93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:11.867086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac176d-a7e8-4bb4-8cac-337593e7ab93 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:11.890642 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 598/1205] 10.222.0.44 () {60 vars in 1132 bytes} [Mon May 9 04:16:11 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:16:14.472420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648879e-5f6e-4ef5-8ae0-7b235e4c03c8 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:14.472970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648879e-5f6e-4ef5-8ae0-7b235e4c03c8 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:14.472970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648879e-5f6e-4ef5-8ae0-7b235e4c03c8 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:14.474330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 608/1206] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:16:14.480031 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92e5d9-382f-4256-89dd-f76224b868ed None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:14.480440 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92e5d9-382f-4256-89dd-f76224b868ed None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:14.480616 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92e5d9-382f-4256-89dd-f76224b868ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:14.499059 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-5c92e5d9-382f-4256-89dd-f76224b868ed None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:16:14.500386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5c92e5d9-382f-4256-89dd-f76224b868ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:16:14.575707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 599/1207] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:16:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:16:14.598322 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9648879e-5f6e-4ef5-8ae0-7b235e4c03c8 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:14.717192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9648879e-5f6e-4ef5-8ae0-7b235e4c03c8 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:14.761186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffb3e1-30bf-4cde-a68b-f8a1a54bac2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:14.762001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffb3e1-30bf-4cde-a68b-f8a1a54bac2a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:14.762285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffb3e1-30bf-4cde-a68b-f8a1a54bac2a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:14.762585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffb3e1-30bf-4cde-a68b-f8a1a54bac2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:14.777233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffb3e1-30bf-4cde-a68b-f8a1a54bac2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-9cffb3e1-30bf-4cde-a68b-f8a1a54bac2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:16:14.787600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:14.788656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:14.792190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:14.792190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:14.792190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:14.792190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:14.792190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:16:14.792190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:16:14.792190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:16:14.792190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:16:14.792190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:16:14.792190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:16:14.792190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:16:14.792190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:14.792190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 609/1208] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:16:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:16:14.796118 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92e5d9-382f-4256-89dd-f76224b868ed None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:14.902545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92e5d9-382f-4256-89dd-f76224b868ed None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:14.942524 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51436c28-5169-4b3b-9545-a05c33bff820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:14.943217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51436c28-5169-4b3b-9545-a05c33bff820 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:14.943538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51436c28-5169-4b3b-9545-a05c33bff820 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:14.943815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51436c28-5169-4b3b-9545-a05c33bff820 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:14.944662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51436c28-5169-4b3b-9545-a05c33bff820 None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:14.944970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51436c28-5169-4b3b-9545-a05c33bff820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:14.946380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51436c28-5169-4b3b-9545-a05c33bff820 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:14.955830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 600/1209] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:16:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:14.961732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffb3e1-30bf-4cde-a68b-f8a1a54bac2a None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:15.083708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffb3e1-30bf-4cde-a68b-f8a1a54bac2a None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:15.142467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d84424-c8de-46d0-855b-f0b11bfcb218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:15.143659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d84424-c8de-46d0-855b-f0b11bfcb218 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:15.144151 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d84424-c8de-46d0-855b-f0b11bfcb218 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:15.144570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d84424-c8de-46d0-855b-f0b11bfcb218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:15.157675 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d84424-c8de-46d0-855b-f0b11bfcb218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:15.159184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d84424-c8de-46d0-855b-f0b11bfcb218 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-d9d84424-c8de-46d0-855b-f0b11bfcb218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:16:15.180601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:15.182097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:15.182995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:16:15.182995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:16:15.182995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:15.182995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:15.183596 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 610/1210] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:15.189410 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51436c28-5169-4b3b-9545-a05c33bff820 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:15.299586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51436c28-5169-4b3b-9545-a05c33bff820 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:15.340594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c5210-bf03-44fc-8683-6154e17822d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:15.341225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c5210-bf03-44fc-8683-6154e17822d6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:15.341484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c5210-bf03-44fc-8683-6154e17822d6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:15.341742 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c5210-bf03-44fc-8683-6154e17822d6 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:15.342475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c5210-bf03-44fc-8683-6154e17822d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:15.342763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c5210-bf03-44fc-8683-6154e17822d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:15.344142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c5210-bf03-44fc-8683-6154e17822d6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:15.359015 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 601/1211] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:15.365199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d84424-c8de-46d0-855b-f0b11bfcb218 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:15.474279 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d84424-c8de-46d0-855b-f0b11bfcb218 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:15.516307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3982c8b-fcc3-444d-a2f4-a34f547ee02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:15.517547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3982c8b-fcc3-444d-a2f4-a34f547ee02c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:15.520113 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3982c8b-fcc3-444d-a2f4-a34f547ee02c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:15.520402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3982c8b-fcc3-444d-a2f4-a34f547ee02c None admin] PATH_INFO: `/v3/users/placement` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:15.529243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3982c8b-fcc3-444d-a2f4-a34f547ee02c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:15.530593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3982c8b-fcc3-444d-a2f4-a34f547ee02c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-a3982c8b-fcc3-444d-a2f4-a34f547ee02c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:16:15.546050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:15.547243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:15.548339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:15.548339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:15.548339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:16:15.548339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:16:15.548339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:16:15.548339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:16:15.548339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:16:15.548339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:16:15.548339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 04:16:15.548339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:15.548339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 611/1212] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:16:15.553195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c5210-bf03-44fc-8683-6154e17822d6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:15.665826 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c5210-bf03-44fc-8683-6154e17822d6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:15.704169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a1270-dc6a-4207-a859-6fd960234db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:15.704912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a1270-dc6a-4207-a859-6fd960234db9 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:15.705452 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a1270-dc6a-4207-a859-6fd960234db9 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:15.705719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a1270-dc6a-4207-a859-6fd960234db9 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:15.706604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a1270-dc6a-4207-a859-6fd960234db9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:15.706933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a1270-dc6a-4207-a859-6fd960234db9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:15.708604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a1270-dc6a-4207-a859-6fd960234db9 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:15.732712 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 602/1213] 10.222.0.44 () {56 vars in 1143 bytes} [Mon May 9 04:16:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:16:15.741993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3982c8b-fcc3-444d-a2f4-a34f547ee02c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:15.864138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3982c8b-fcc3-444d-a2f4-a34f547ee02c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:15.901871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e678a-d4b2-49c9-a983-eebfa65d0178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:15.902609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e678a-d4b2-49c9-a983-eebfa65d0178 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:15.902931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e678a-d4b2-49c9-a983-eebfa65d0178 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:15.903217 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e678a-d4b2-49c9-a983-eebfa65d0178 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:15.979682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e678a-d4b2-49c9-a983-eebfa65d0178 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:15.981318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e678a-d4b2-49c9-a983-eebfa65d0178 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:16.101895 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 612/1214] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:16:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 04:16:16.107925 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a1270-dc6a-4207-a859-6fd960234db9 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:16.220395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a1270-dc6a-4207-a859-6fd960234db9 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:16.257771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499a113-b6ea-4d14-88f6-45e1c48f4389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:16.258544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499a113-b6ea-4d14-88f6-45e1c48f4389 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:16.258853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499a113-b6ea-4d14-88f6-45e1c48f4389 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:16.259138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499a113-b6ea-4d14-88f6-45e1c48f4389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:16.271947 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499a113-b6ea-4d14-88f6-45e1c48f4389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:16.273500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499a113-b6ea-4d14-88f6-45e1c48f4389 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-0499a113-b6ea-4d14-88f6-45e1c48f4389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:16:16.296607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:16.297843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:16.298979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:16:16.298979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:16:16.298979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:16.298979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:16.304036 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 603/1215] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:16:16.305230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e678a-d4b2-49c9-a983-eebfa65d0178 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:16.415565 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e678a-d4b2-49c9-a983-eebfa65d0178 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:16.458834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6af444-a048-4e88-9ae4-d58734f2f7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:16.459792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6af444-a048-4e88-9ae4-d58734f2f7d9 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:16.460258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6af444-a048-4e88-9ae4-d58734f2f7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:16.460707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6af444-a048-4e88-9ae4-d58734f2f7d9 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:16.461620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6af444-a048-4e88-9ae4-d58734f2f7d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:16.462085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6af444-a048-4e88-9ae4-d58734f2f7d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:16.464433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6af444-a048-4e88-9ae4-d58734f2f7d9 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:16.477237 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 613/1216] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:16.482756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499a113-b6ea-4d14-88f6-45e1c48f4389 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:16.589629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499a113-b6ea-4d14-88f6-45e1c48f4389 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:16.625033 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fb40e-800d-4fa1-aeee-8353a32eabeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:16.625833 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fb40e-800d-4fa1-aeee-8353a32eabeb None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:16.626108 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fb40e-800d-4fa1-aeee-8353a32eabeb None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:16.626377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fb40e-800d-4fa1-aeee-8353a32eabeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:16.633078 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fb40e-800d-4fa1-aeee-8353a32eabeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:16.634520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fb40e-800d-4fa1-aeee-8353a32eabeb None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-417fb40e-800d-4fa1-aeee-8353a32eabeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:16:16.648536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:16.649554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:16.651045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:16:16.651045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:16:16.651045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:16:16.651045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:16:16.651045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:16:16.651045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:16:16.651045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:16:16.651045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:16.651045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 604/1217] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:16:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:16.656326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6af444-a048-4e88-9ae4-d58734f2f7d9 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:16.759514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6af444-a048-4e88-9ae4-d58734f2f7d9 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:16.795424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99829edf-185e-4e7b-80f9-09a5c01738c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:16.796106 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99829edf-185e-4e7b-80f9-09a5c01738c9 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:16.796383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99829edf-185e-4e7b-80f9-09a5c01738c9 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:16.796653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99829edf-185e-4e7b-80f9-09a5c01738c9 None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:16.797433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99829edf-185e-4e7b-80f9-09a5c01738c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:16.797736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99829edf-185e-4e7b-80f9-09a5c01738c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:16.799141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99829edf-185e-4e7b-80f9-09a5c01738c9 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:16.817724 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fb40e-800d-4fa1-aeee-8353a32eabeb None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:16.818161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 614/1218] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:16:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:16.922985 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fb40e-800d-4fa1-aeee-8353a32eabeb None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:16.959320 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163633f-ed1d-49b2-8f01-03427b26111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:16.960045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163633f-ed1d-49b2-8f01-03427b26111a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:16.960359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163633f-ed1d-49b2-8f01-03427b26111a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:16.960649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163633f-ed1d-49b2-8f01-03427b26111a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:16.961545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163633f-ed1d-49b2-8f01-03427b26111a None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=d41e05b7ef954dd596c0134b65434dd6) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:16.961856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163633f-ed1d-49b2-8f01-03427b26111a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:16.963494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163633f-ed1d-49b2-8f01-03427b26111a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:16.969556 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 605/1219] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:16:16 2022] GET /identity/v3/role_assignments?user.id=d41e05b7ef954dd596c0134b65434dd6&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:19.232060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915b1cb-fa3c-495e-a92e-eccfdd21b403 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:19.232643 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915b1cb-fa3c-495e-a92e-eccfdd21b403 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:19.232643 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915b1cb-fa3c-495e-a92e-eccfdd21b403 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:19.234068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 615/1220] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:16:19.239793 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de720e8-9f38-4681-9a73-9050ebf5ff3a None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:19.240270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de720e8-9f38-4681-9a73-9050ebf5ff3a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:19.240422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de720e8-9f38-4681-9a73-9050ebf5ff3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:19.258609 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-5de720e8-9f38-4681-9a73-9050ebf5ff3a None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:16:19.259810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5de720e8-9f38-4681-9a73-9050ebf5ff3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:16:19.329101 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 606/1221] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:16:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:16:19.351571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915b1cb-fa3c-495e-a92e-eccfdd21b403 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:19.458426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915b1cb-fa3c-495e-a92e-eccfdd21b403 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:19.506592 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dc35f-6986-4d91-9a11-040efe81c514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:19.507247 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dc35f-6986-4d91-9a11-040efe81c514 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:19.507543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dc35f-6986-4d91-9a11-040efe81c514 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:19.507875 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dc35f-6986-4d91-9a11-040efe81c514 None admin] PATH_INFO: `/v3/roles/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:19.522290 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352dc35f-6986-4d91-9a11-040efe81c514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-352dc35f-6986-4d91-9a11-040efe81c514 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:16:19.530788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:19.531791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:19.532665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:19.532665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:19.532665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:19.532665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:19.532665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:16:19.532665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:16:19.532665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:16:19.532665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:16:19.532665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:16:19.532665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:16:19.532665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:16:19.532665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:19.537041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 616/1222] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:16:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:19.545317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de720e8-9f38-4681-9a73-9050ebf5ff3a None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:19.640130 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de720e8-9f38-4681-9a73-9050ebf5ff3a None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:19.676460 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a61ba-8ed7-442c-91ad-3c7ff3af3ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:19.677188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a61ba-8ed7-442c-91ad-3c7ff3af3ef5 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:19.677536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a61ba-8ed7-442c-91ad-3c7ff3af3ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:19.677815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a61ba-8ed7-442c-91ad-3c7ff3af3ef5 None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:19.678684 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a61ba-8ed7-442c-91ad-3c7ff3af3ef5 None admin] RBAC: Adding query filter params (name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:19.679049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a61ba-8ed7-442c-91ad-3c7ff3af3ef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:19.680547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a61ba-8ed7-442c-91ad-3c7ff3af3ef5 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:19.688935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 607/1223] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:16:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:19.695373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dc35f-6986-4d91-9a11-040efe81c514 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:19.805795 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dc35f-6986-4d91-9a11-040efe81c514 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:19.838752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddde1a3-e78c-49a6-9ef6-c120c9de36ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:19.839548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddde1a3-e78c-49a6-9ef6-c120c9de36ca None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:19.839828 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddde1a3-e78c-49a6-9ef6-c120c9de36ca None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:19.840129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddde1a3-e78c-49a6-9ef6-c120c9de36ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:19.850135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddde1a3-e78c-49a6-9ef6-c120c9de36ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:19.851576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddde1a3-e78c-49a6-9ef6-c120c9de36ca None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-3ddde1a3-e78c-49a6-9ef6-c120c9de36ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:16:19.870119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:19.871433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:19.872503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:16:19.872503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:16:19.872503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:19.872503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:19.872503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 617/1224] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:16:19.877352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a61ba-8ed7-442c-91ad-3c7ff3af3ef5 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:19.979445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a61ba-8ed7-442c-91ad-3c7ff3af3ef5 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:20.013533 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3dd1d7-5618-41f4-a2c0-11141b782945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:20.014193 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3dd1d7-5618-41f4-a2c0-11141b782945 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:20.014476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3dd1d7-5618-41f4-a2c0-11141b782945 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:20.014756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3dd1d7-5618-41f4-a2c0-11141b782945 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:20.015468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3dd1d7-5618-41f4-a2c0-11141b782945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:20.015770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3dd1d7-5618-41f4-a2c0-11141b782945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:20.017161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3dd1d7-5618-41f4-a2c0-11141b782945 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:20.028581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 608/1225] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:20.034384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddde1a3-e78c-49a6-9ef6-c120c9de36ca None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:20.148777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddde1a3-e78c-49a6-9ef6-c120c9de36ca None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:20.183396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4243-b6cf-4efe-a168-ae5cb9bf289c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:20.184223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4243-b6cf-4efe-a168-ae5cb9bf289c None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:20.184505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4243-b6cf-4efe-a168-ae5cb9bf289c None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:20.184776 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4243-b6cf-4efe-a168-ae5cb9bf289c None admin] PATH_INFO: `/v3/users/placement` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:20.192378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b4243-b6cf-4efe-a168-ae5cb9bf289c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:20.193803 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b4243-b6cf-4efe-a168-ae5cb9bf289c None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-d23b4243-b6cf-4efe-a168-ae5cb9bf289c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:16:20.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:20.210009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:20.211180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:20.211180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:20.211180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:16:20.211180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:16:20.211180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:16:20.211180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:16:20.211180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:16:20.211180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:16:20.211180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 04:16:20.211180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:20.211180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 618/1226] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:20.216172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3dd1d7-5618-41f4-a2c0-11141b782945 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:20.318028 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3dd1d7-5618-41f4-a2c0-11141b782945 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:20.357313 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b218f9d-4abf-416d-91d2-69b972104efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:20.357966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b218f9d-4abf-416d-91d2-69b972104efc None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:20.358250 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b218f9d-4abf-416d-91d2-69b972104efc None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:20.358530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b218f9d-4abf-416d-91d2-69b972104efc None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:20.359489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b218f9d-4abf-416d-91d2-69b972104efc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:20.359829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b218f9d-4abf-416d-91d2-69b972104efc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:20.361334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b218f9d-4abf-416d-91d2-69b972104efc None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:20.379380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 609/1227] 10.222.0.44 () {56 vars in 1143 bytes} [Mon May 9 04:16:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:20.385882 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4243-b6cf-4efe-a168-ae5cb9bf289c None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:20.503275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4243-b6cf-4efe-a168-ae5cb9bf289c None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:20.538480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44891cea-755f-41b9-a27d-85e53ab5a020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:20.539282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44891cea-755f-41b9-a27d-85e53ab5a020 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:20.539559 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44891cea-755f-41b9-a27d-85e53ab5a020 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:20.539831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44891cea-755f-41b9-a27d-85e53ab5a020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:20.550784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44891cea-755f-41b9-a27d-85e53ab5a020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:20.552236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44891cea-755f-41b9-a27d-85e53ab5a020 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-44891cea-755f-41b9-a27d-85e53ab5a020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:16:20.569296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:20.570483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:20.571682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:16:20.571682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:16:20.571682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:20.571682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:20.571682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 619/1228] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:20.577005 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b218f9d-4abf-416d-91d2-69b972104efc None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:20.678605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b218f9d-4abf-416d-91d2-69b972104efc None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:20.715747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a8254-8ba1-4f75-8fe7-d74bb1e9b80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:20.716434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a8254-8ba1-4f75-8fe7-d74bb1e9b80b None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:20.716723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a8254-8ba1-4f75-8fe7-d74bb1e9b80b None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:20.716984 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a8254-8ba1-4f75-8fe7-d74bb1e9b80b None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:20.717694 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a8254-8ba1-4f75-8fe7-d74bb1e9b80b None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:20.717990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a8254-8ba1-4f75-8fe7-d74bb1e9b80b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:20.719337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a8254-8ba1-4f75-8fe7-d74bb1e9b80b None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:20.731157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 610/1229] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:20.737008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44891cea-755f-41b9-a27d-85e53ab5a020 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:20.846466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44891cea-755f-41b9-a27d-85e53ab5a020 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:20.886516 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc1be1-3f72-4e81-89f8-83918938caac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:20.887282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc1be1-3f72-4e81-89f8-83918938caac None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:20.887569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc1be1-3f72-4e81-89f8-83918938caac None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:20.887840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc1be1-3f72-4e81-89f8-83918938caac None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:20.894666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc1be1-3f72-4e81-89f8-83918938caac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:20.896136 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc1be1-3f72-4e81-89f8-83918938caac None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-aecc1be1-3f72-4e81-89f8-83918938caac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:16:20.909927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:20.911026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:20.912284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:16:20.912284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:16:20.912284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:16:20.912284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:16:20.912284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:16:20.912284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:16:20.912284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:16:20.912284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:20.912284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 620/1230] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:16:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:20.917537 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a8254-8ba1-4f75-8fe7-d74bb1e9b80b None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:21.019878 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a8254-8ba1-4f75-8fe7-d74bb1e9b80b None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:21.055791 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76578189-c879-4212-a037-5f84c7c5cca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:21.057238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578189-c879-4212-a037-5f84c7c5cca6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:21.057591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578189-c879-4212-a037-5f84c7c5cca6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:21.057968 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578189-c879-4212-a037-5f84c7c5cca6 None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:21.058840 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76578189-c879-4212-a037-5f84c7c5cca6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:21.059147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76578189-c879-4212-a037-5f84c7c5cca6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:21.060800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76578189-c879-4212-a037-5f84c7c5cca6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:21.073568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 611/1231] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:16:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:16:21.079496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc1be1-3f72-4e81-89f8-83918938caac None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:21.184634 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc1be1-3f72-4e81-89f8-83918938caac None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:21.221839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99681b93-3679-48fa-8745-aad046030b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:21.222736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99681b93-3679-48fa-8745-aad046030b91 None admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:21.223011 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99681b93-3679-48fa-8745-aad046030b91 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:21.223280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99681b93-3679-48fa-8745-aad046030b91 None admin] PATH_INFO: `/v3/projects/3322b00723974bd99e837d447bac22d1/users/d41e05b7ef954dd596c0134b65434dd6/roles/c255c186230d4339b46d5e1f7f1f6cbc` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:21.238841 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99681b93-3679-48fa-8745-aad046030b91 None admin] RBAC: Authorizing `identity:create_grant(project_id=3322b00723974bd99e837d447bac22d1, user_id=d41e05b7ef954dd596c0134b65434dd6, role_id=c255c186230d4339b46d5e1f7f1f6cbc)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:21.240654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99681b93-3679-48fa-8745-aad046030b91 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:21.253665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 621/1232] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:16:21 2022] PUT /identity/v3/projects/3322b00723974bd99e837d447bac22d1/users/d41e05b7ef954dd596c0134b65434dd6/roles/c255c186230d4339b46d5e1f7f1f6cbc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:16:23.480972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca8651-d861-4fd7-8890-fec0dd6329a2 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:23.481522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca8651-d861-4fd7-8890-fec0dd6329a2 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:23.481615 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca8651-d861-4fd7-8890-fec0dd6329a2 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:23.487155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 612/1233] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:16:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:16:23.488900 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950d809-a76b-4003-b144-518899fe28fa None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:23.489359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950d809-a76b-4003-b144-518899fe28fa None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:23.489453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950d809-a76b-4003-b144-518899fe28fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:23.505972 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-0950d809-a76b-4003-b144-518899fe28fa None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:16:23.507135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-0950d809-a76b-4003-b144-518899fe28fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:16:23.576836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 622/1234] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:16:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:16:23.600723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca8651-d861-4fd7-8890-fec0dd6329a2 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:23.710216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca8651-d861-4fd7-8890-fec0dd6329a2 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:23.751012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787bd5a-a4ec-4794-9c32-b8065a43bfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:23.751738 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787bd5a-a4ec-4794-9c32-b8065a43bfdb None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:23.752056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787bd5a-a4ec-4794-9c32-b8065a43bfdb None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:23.752351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787bd5a-a4ec-4794-9c32-b8065a43bfdb None admin] PATH_INFO: `/v3/roles/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:23.763476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787bd5a-a4ec-4794-9c32-b8065a43bfdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-f787bd5a-a4ec-4794-9c32-b8065a43bfdb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:16:23.772706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:23.774008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:23.774979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:23.774979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:23.774979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:23.774979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:23.774979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:16:23.774979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:16:23.774979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:16:23.774979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:16:23.774979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:16:23.774979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:16:23.774979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 04:16:23.774979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:23.774979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 613/1235] 10.222.0.44 () {56 vars in 1102 bytes} [Mon May 9 04:16:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:23.780824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950d809-a76b-4003-b144-518899fe28fa None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:23.887280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950d809-a76b-4003-b144-518899fe28fa None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:23.923927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eac4ea-fff1-4b5d-a5b0-815fc81b1a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:23.923927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eac4ea-fff1-4b5d-a5b0-815fc81b1a78 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:23.923927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eac4ea-fff1-4b5d-a5b0-815fc81b1a78 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:23.923927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eac4ea-fff1-4b5d-a5b0-815fc81b1a78 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:23.924435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eac4ea-fff1-4b5d-a5b0-815fc81b1a78 None admin] RBAC: Adding query filter params (name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:23.924435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eac4ea-fff1-4b5d-a5b0-815fc81b1a78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:23.924863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eac4ea-fff1-4b5d-a5b0-815fc81b1a78 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:23.935503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 623/1236] 10.222.0.44 () {56 vars in 1103 bytes} [Mon May 9 04:16:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:23.941464 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787bd5a-a4ec-4794-9c32-b8065a43bfdb None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:24.049903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787bd5a-a4ec-4794-9c32-b8065a43bfdb None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:24.083473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a921e8-d324-4a83-9ee7-03e854111c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:24.084383 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a921e8-d324-4a83-9ee7-03e854111c63 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:24.084669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a921e8-d324-4a83-9ee7-03e854111c63 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:24.085076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a921e8-d324-4a83-9ee7-03e854111c63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:24.095959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a921e8-d324-4a83-9ee7-03e854111c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:24.097406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a921e8-d324-4a83-9ee7-03e854111c63 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-f9a921e8-d324-4a83-9ee7-03e854111c63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:16:24.117678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:24.118963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:24.119969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:16:24.119969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:16:24.119969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:24.119969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:24.119969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 614/1237] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:24.124908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eac4ea-fff1-4b5d-a5b0-815fc81b1a78 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:24.226811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eac4ea-fff1-4b5d-a5b0-815fc81b1a78 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:24.265566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9974a4f-ee57-4c9a-a3e4-b56dda1e323d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:24.266467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9974a4f-ee57-4c9a-a3e4-b56dda1e323d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:24.266905 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9974a4f-ee57-4c9a-a3e4-b56dda1e323d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:24.267325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9974a4f-ee57-4c9a-a3e4-b56dda1e323d None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:24.268259 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9974a4f-ee57-4c9a-a3e4-b56dda1e323d None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:24.268683 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9974a4f-ee57-4c9a-a3e4-b56dda1e323d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:24.270279 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9974a4f-ee57-4c9a-a3e4-b56dda1e323d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:24.286332 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 624/1238] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:16:24.287503 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a921e8-d324-4a83-9ee7-03e854111c63 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:24.388557 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a921e8-d324-4a83-9ee7-03e854111c63 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:24.423678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428f867-c251-4610-b5d9-e17acf67465c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:24.424514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428f867-c251-4610-b5d9-e17acf67465c None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:24.424802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428f867-c251-4610-b5d9-e17acf67465c None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:24.425071 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428f867-c251-4610-b5d9-e17acf67465c None admin] PATH_INFO: `/v3/users/placement` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:24.433192 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428f867-c251-4610-b5d9-e17acf67465c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:24.434629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428f867-c251-4610-b5d9-e17acf67465c None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-1428f867-c251-4610-b5d9-e17acf67465c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:16:24.448976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:24.450201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:24.451242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:24.451242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:24.451242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:16:24.451242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:16:24.451242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:16:24.451242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:16:24.451242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:16:24.451242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:16:24.451242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 04:16:24.451242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:24.451242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 615/1239] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:24.456314 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9974a4f-ee57-4c9a-a3e4-b56dda1e323d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:24.570891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9974a4f-ee57-4c9a-a3e4-b56dda1e323d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:24.610598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a49096-8de9-4c75-a88a-21f722d13604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:24.611425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a49096-8de9-4c75-a88a-21f722d13604 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:24.611740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a49096-8de9-4c75-a88a-21f722d13604 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:24.612092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a49096-8de9-4c75-a88a-21f722d13604 None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:24.612967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a49096-8de9-4c75-a88a-21f722d13604 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:24.613362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a49096-8de9-4c75-a88a-21f722d13604 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:24.614890 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a49096-8de9-4c75-a88a-21f722d13604 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:24.633790 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 625/1240] 10.222.0.44 () {56 vars in 1143 bytes} [Mon May 9 04:16:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:24.640194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428f867-c251-4610-b5d9-e17acf67465c None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:24.741755 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428f867-c251-4610-b5d9-e17acf67465c None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:24.777562 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d30be-e1f5-45ee-8f13-df4d9d606f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:24.778277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d30be-e1f5-45ee-8f13-df4d9d606f90 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:24.778500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d30be-e1f5-45ee-8f13-df4d9d606f90 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:24.778797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d30be-e1f5-45ee-8f13-df4d9d606f90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:24.856403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d30be-e1f5-45ee-8f13-df4d9d606f90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:24.857853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d30be-e1f5-45ee-8f13-df4d9d606f90 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:24.957905 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 616/1241] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:16:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 04:16:24.963571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a49096-8de9-4c75-a88a-21f722d13604 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:25.067133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a49096-8de9-4c75-a88a-21f722d13604 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:25.105499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5839ea0-2b2e-4c20-b1f2-682e6f210617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:25.106312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5839ea0-2b2e-4c20-b1f2-682e6f210617 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:25.106594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5839ea0-2b2e-4c20-b1f2-682e6f210617 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:25.106864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5839ea0-2b2e-4c20-b1f2-682e6f210617 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:25.117819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5839ea0-2b2e-4c20-b1f2-682e6f210617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:25.120241 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5839ea0-2b2e-4c20-b1f2-682e6f210617 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-a5839ea0-2b2e-4c20-b1f2-682e6f210617 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:16:25.137919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:25.139270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:25.140392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:16:25.140392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:16:25.140392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:25.140392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:25.140392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 626/1242] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:25.145143 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d30be-e1f5-45ee-8f13-df4d9d606f90 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:25.246210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d30be-e1f5-45ee-8f13-df4d9d606f90 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:25.279771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b146ea2-63b9-400a-a223-3c20b909ec91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:25.280534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b146ea2-63b9-400a-a223-3c20b909ec91 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:25.280809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b146ea2-63b9-400a-a223-3c20b909ec91 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:25.281083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b146ea2-63b9-400a-a223-3c20b909ec91 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:25.281792 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b146ea2-63b9-400a-a223-3c20b909ec91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:25.282096 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b146ea2-63b9-400a-a223-3c20b909ec91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:25.283440 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b146ea2-63b9-400a-a223-3c20b909ec91 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:25.294730 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 617/1243] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:25.300695 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5839ea0-2b2e-4c20-b1f2-682e6f210617 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:25.397840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5839ea0-2b2e-4c20-b1f2-682e6f210617 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:25.432003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616e3ac-dc29-448f-93ce-c2e23302c77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:25.432799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616e3ac-dc29-448f-93ce-c2e23302c77b None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:25.433095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616e3ac-dc29-448f-93ce-c2e23302c77b None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:25.433343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616e3ac-dc29-448f-93ce-c2e23302c77b None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:25.440533 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616e3ac-dc29-448f-93ce-c2e23302c77b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:25.441980 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616e3ac-dc29-448f-93ce-c2e23302c77b None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-6616e3ac-dc29-448f-93ce-c2e23302c77b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:16:25.458576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:25.459704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:25.460863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:16:25.460863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:16:25.460863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:16:25.460863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:16:25.460863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:16:25.460863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:16:25.460863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:16:25.460863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:25.460863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 627/1244] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:16:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:16:25.466443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b146ea2-63b9-400a-a223-3c20b909ec91 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:25.580560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b146ea2-63b9-400a-a223-3c20b909ec91 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:25.618717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a5222-1b94-43ac-9d33-3560b26c5fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:25.620032 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a5222-1b94-43ac-9d33-3560b26c5fde None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:25.620322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a5222-1b94-43ac-9d33-3560b26c5fde None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:25.620608 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a5222-1b94-43ac-9d33-3560b26c5fde None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:25.621538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a5222-1b94-43ac-9d33-3560b26c5fde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:25.622026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a5222-1b94-43ac-9d33-3560b26c5fde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:25.623868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a5222-1b94-43ac-9d33-3560b26c5fde None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:25.638650 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 618/1245] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:16:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:25.644668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616e3ac-dc29-448f-93ce-c2e23302c77b None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:25.744594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616e3ac-dc29-448f-93ce-c2e23302c77b None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:25.781830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576db66-5e89-4a05-b669-4e509a090dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:25.782520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576db66-5e89-4a05-b669-4e509a090dfa None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:25.782797 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576db66-5e89-4a05-b669-4e509a090dfa None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:25.783070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576db66-5e89-4a05-b669-4e509a090dfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:25.783914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576db66-5e89-4a05-b669-4e509a090dfa None admin] RBAC: Adding query filter params (role.id=c255c186230d4339b46d5e1f7f1f6cbc, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=d41e05b7ef954dd596c0134b65434dd6) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:25.784238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576db66-5e89-4a05-b669-4e509a090dfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:25.785690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576db66-5e89-4a05-b669-4e509a090dfa None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:25.791980 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 628/1246] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:16:25 2022] GET /identity/v3/role_assignments?user.id=d41e05b7ef954dd596c0134b65434dd6&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=c255c186230d4339b46d5e1f7f1f6cbc => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:28.126228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a43f94-4305-4383-a890-9ca53b0b6f9a None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:28.126741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a43f94-4305-4383-a890-9ca53b0b6f9a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:28.126741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a43f94-4305-4383-a890-9ca53b0b6f9a None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:28.128087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 619/1247] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:16:28.135172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621f5bd-3780-446c-bfed-ee3337638ec4 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:28.135172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621f5bd-3780-446c-bfed-ee3337638ec4 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:28.135172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621f5bd-3780-446c-bfed-ee3337638ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:28.153234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-8621f5bd-3780-446c-bfed-ee3337638ec4 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:16:28.154508 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8621f5bd-3780-446c-bfed-ee3337638ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:16:28.225812 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 629/1248] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:16:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:16:28.250644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a43f94-4305-4383-a890-9ca53b0b6f9a None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:28.361344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a43f94-4305-4383-a890-9ca53b0b6f9a None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:28.401047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14388f-fa1d-42ec-a233-b84ef88c962a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:28.402028 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14388f-fa1d-42ec-a233-b84ef88c962a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:28.402376 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14388f-fa1d-42ec-a233-b84ef88c962a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:28.402719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14388f-fa1d-42ec-a233-b84ef88c962a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:28.413833 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14388f-fa1d-42ec-a233-b84ef88c962a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-fa14388f-fa1d-42ec-a233-b84ef88c962a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:16:28.423121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:28.424303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:28.425299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:28.425299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:28.425299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:28.425299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:28.425299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:16:28.425299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:16:28.425299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:16:28.425299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:16:28.425299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:16:28.425299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:16:28.425299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:16:28.425299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:28.425299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 620/1249] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:16:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:28.430356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621f5bd-3780-446c-bfed-ee3337638ec4 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:28.533671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621f5bd-3780-446c-bfed-ee3337638ec4 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:28.569008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb359b43-3884-477d-b576-c0aa9753f972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:28.569838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb359b43-3884-477d-b576-c0aa9753f972 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:28.570195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb359b43-3884-477d-b576-c0aa9753f972 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:28.570517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb359b43-3884-477d-b576-c0aa9753f972 None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:28.571470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb359b43-3884-477d-b576-c0aa9753f972 None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:28.571833 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb359b43-3884-477d-b576-c0aa9753f972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:28.574145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb359b43-3884-477d-b576-c0aa9753f972 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:28.584254 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 630/1250] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:16:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:28.589845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14388f-fa1d-42ec-a233-b84ef88c962a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:28.695075 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14388f-fa1d-42ec-a233-b84ef88c962a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:28.732244 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809240c-66ef-4acf-ad8b-21c2f060ba1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:28.733041 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809240c-66ef-4acf-ad8b-21c2f060ba1a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:28.733326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809240c-66ef-4acf-ad8b-21c2f060ba1a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:28.733601 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809240c-66ef-4acf-ad8b-21c2f060ba1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:28.744697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809240c-66ef-4acf-ad8b-21c2f060ba1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:28.746229 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809240c-66ef-4acf-ad8b-21c2f060ba1a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-d809240c-66ef-4acf-ad8b-21c2f060ba1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:16:28.764116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:28.765126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:28.766220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:16:28.766220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:16:28.766220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:28.766220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:28.766220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 621/1251] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:28.771394 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb359b43-3884-477d-b576-c0aa9753f972 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:28.893566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb359b43-3884-477d-b576-c0aa9753f972 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:28.925818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd4635-be43-49e8-8e20-062d2cae01de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:28.926476 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd4635-be43-49e8-8e20-062d2cae01de None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:28.926753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd4635-be43-49e8-8e20-062d2cae01de None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:28.927024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd4635-be43-49e8-8e20-062d2cae01de None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:28.927739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd4635-be43-49e8-8e20-062d2cae01de None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:28.928071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd4635-be43-49e8-8e20-062d2cae01de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:28.929433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd4635-be43-49e8-8e20-062d2cae01de None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:28.942283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 631/1252] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:16:28.948493 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809240c-66ef-4acf-ad8b-21c2f060ba1a None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:29.055730 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809240c-66ef-4acf-ad8b-21c2f060ba1a None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:29.095837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b169b31-ff18-4605-8141-5e47b581c0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:29.096909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b169b31-ff18-4605-8141-5e47b581c0bf None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:29.097245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b169b31-ff18-4605-8141-5e47b581c0bf None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:29.097468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b169b31-ff18-4605-8141-5e47b581c0bf None admin] PATH_INFO: `/v3/users/placement` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:29.106140 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b169b31-ff18-4605-8141-5e47b581c0bf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:29.107602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b169b31-ff18-4605-8141-5e47b581c0bf None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-1b169b31-ff18-4605-8141-5e47b581c0bf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:16:29.121746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:29.123003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:29.124196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:29.124196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:29.124196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:16:29.124196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:16:29.124196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:16:29.124196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:16:29.124196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:16:29.124196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:16:29.124196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 04:16:29.124196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:29.124196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 622/1253] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:16:29.130101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd4635-be43-49e8-8e20-062d2cae01de None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:29.232158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd4635-be43-49e8-8e20-062d2cae01de None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:29.266327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e36d4c-f188-4705-a06d-3993342cb5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:29.267000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36d4c-f188-4705-a06d-3993342cb5fa None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:29.267307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36d4c-f188-4705-a06d-3993342cb5fa None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:29.267607 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36d4c-f188-4705-a06d-3993342cb5fa None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:29.268456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e36d4c-f188-4705-a06d-3993342cb5fa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:29.268765 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e36d4c-f188-4705-a06d-3993342cb5fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:29.270196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e36d4c-f188-4705-a06d-3993342cb5fa None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:29.288439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 632/1254] 10.222.0.44 () {56 vars in 1143 bytes} [Mon May 9 04:16:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:29.294162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b169b31-ff18-4605-8141-5e47b581c0bf None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:29.394304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b169b31-ff18-4605-8141-5e47b581c0bf None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:29.426530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee821e-5045-4ba1-a08f-421e2aa7bd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:29.427183 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee821e-5045-4ba1-a08f-421e2aa7bd53 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:29.427466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee821e-5045-4ba1-a08f-421e2aa7bd53 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:29.427743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee821e-5045-4ba1-a08f-421e2aa7bd53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:29.499190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee821e-5045-4ba1-a08f-421e2aa7bd53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:29.500772 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee821e-5045-4ba1-a08f-421e2aa7bd53 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:29.602542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 623/1255] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:16:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:16:29.609351 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e36d4c-f188-4705-a06d-3993342cb5fa None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:29.715586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e36d4c-f188-4705-a06d-3993342cb5fa None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:29.751575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6a67b-f940-4351-b417-9c7859bf15f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:29.752465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6a67b-f940-4351-b417-9c7859bf15f0 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:29.752770 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6a67b-f940-4351-b417-9c7859bf15f0 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:29.753108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6a67b-f940-4351-b417-9c7859bf15f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:29.764834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6a67b-f940-4351-b417-9c7859bf15f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:29.767415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6a67b-f940-4351-b417-9c7859bf15f0 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-34d6a67b-f940-4351-b417-9c7859bf15f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:16:29.786285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:29.787380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:29.788448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:16:29.788448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:16:29.788448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:29.788448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:29.790009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 633/1256] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:29.796490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee821e-5045-4ba1-a08f-421e2aa7bd53 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:29.902134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee821e-5045-4ba1-a08f-421e2aa7bd53 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:29.948122 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4a288-7a1a-4818-8ed5-04c09b2d6cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:29.949148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4a288-7a1a-4818-8ed5-04c09b2d6cf6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:29.949630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4a288-7a1a-4818-8ed5-04c09b2d6cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:29.950040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4a288-7a1a-4818-8ed5-04c09b2d6cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:29.951210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4a288-7a1a-4818-8ed5-04c09b2d6cf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:29.951679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4a288-7a1a-4818-8ed5-04c09b2d6cf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:29.953971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4a288-7a1a-4818-8ed5-04c09b2d6cf6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:29.970011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 624/1257] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:16:29.976074 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6a67b-f940-4351-b417-9c7859bf15f0 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:30.077619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6a67b-f940-4351-b417-9c7859bf15f0 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:30.113212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7443d-5575-47ff-9514-4e536fbc3fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:30.114013 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7443d-5575-47ff-9514-4e536fbc3fd0 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:30.114291 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7443d-5575-47ff-9514-4e536fbc3fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:30.114560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7443d-5575-47ff-9514-4e536fbc3fd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:30.122031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7443d-5575-47ff-9514-4e536fbc3fd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:30.123513 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7443d-5575-47ff-9514-4e536fbc3fd0 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-c3f7443d-5575-47ff-9514-4e536fbc3fd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:16:30.138040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:30.139170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:30.140330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:16:30.140330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:16:30.140330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:16:30.140330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:16:30.140330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:16:30.140330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:16:30.140330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:16:30.140330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:30.140330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 634/1258] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:16:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:30.147164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4a288-7a1a-4818-8ed5-04c09b2d6cf6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:30.249629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4a288-7a1a-4818-8ed5-04c09b2d6cf6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:30.284680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae831c-aaa4-49ec-9387-95dfb6d4333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:30.285353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae831c-aaa4-49ec-9387-95dfb6d4333a None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:30.285635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae831c-aaa4-49ec-9387-95dfb6d4333a None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:30.285907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae831c-aaa4-49ec-9387-95dfb6d4333a None admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:30.286702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae831c-aaa4-49ec-9387-95dfb6d4333a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:30.287007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae831c-aaa4-49ec-9387-95dfb6d4333a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:30.288486 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae831c-aaa4-49ec-9387-95dfb6d4333a None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:30.301969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 625/1259] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:16:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:30.308128 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7443d-5575-47ff-9514-4e536fbc3fd0 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:30.405863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7443d-5575-47ff-9514-4e536fbc3fd0 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:30.437941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b033e-0952-455a-89b6-cba9e3481fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:30.438600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b033e-0952-455a-89b6-cba9e3481fe8 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:30.438883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b033e-0952-455a-89b6-cba9e3481fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:30.439159 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b033e-0952-455a-89b6-cba9e3481fe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:30.440106 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b033e-0952-455a-89b6-cba9e3481fe8 None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=d41e05b7ef954dd596c0134b65434dd6) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:30.440419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b033e-0952-455a-89b6-cba9e3481fe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:30.441827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b033e-0952-455a-89b6-cba9e3481fe8 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:30.447763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 635/1260] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:16:30 2022] GET /identity/v3/role_assignments?user.id=d41e05b7ef954dd596c0134b65434dd6&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=e59113c0366049e889df3f55eed90c72 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:32.687187 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382104a-36b1-45ef-8f0f-a2667b6ad53c None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:32.687740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382104a-36b1-45ef-8f0f-a2667b6ad53c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:32.687740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382104a-36b1-45ef-8f0f-a2667b6ad53c None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:32.689125 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 626/1261] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:16:32.694137 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f15e2-4d0b-4482-a4f8-d850177e0e2d None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:32.694533 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f15e2-4d0b-4482-a4f8-d850177e0e2d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:32.694687 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f15e2-4d0b-4482-a4f8-d850177e0e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:32.711074 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e50f15e2-4d0b-4482-a4f8-d850177e0e2d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:16:32.712259 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e50f15e2-4d0b-4482-a4f8-d850177e0e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:16:32.776677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 636/1262] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:16:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:16:32.800201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382104a-36b1-45ef-8f0f-a2667b6ad53c None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:32.907487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382104a-36b1-45ef-8f0f-a2667b6ad53c None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:32.940670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71da988-43e7-4d2c-ba30-082eb666d611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:32.941380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71da988-43e7-4d2c-ba30-082eb666d611 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:32.941660 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71da988-43e7-4d2c-ba30-082eb666d611 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:32.941934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71da988-43e7-4d2c-ba30-082eb666d611 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:32.951681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71da988-43e7-4d2c-ba30-082eb666d611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-b71da988-43e7-4d2c-ba30-082eb666d611 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:16:32.959359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:32.960470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:32.961339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:32.961339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:32.961339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:32.961339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:32.961339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:16:32.961339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:16:32.961339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:16:32.961339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:16:32.961339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:16:32.961339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:16:32.961339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:16:32.961339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:32.961339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 627/1263] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:16:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:32.967017 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f15e2-4d0b-4482-a4f8-d850177e0e2d None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:33.057029 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f15e2-4d0b-4482-a4f8-d850177e0e2d None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:33.088326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c52fb-7ef0-4c4c-b119-f2414fa5dd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:33.088976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c52fb-7ef0-4c4c-b119-f2414fa5dd2d None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:33.089249 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c52fb-7ef0-4c4c-b119-f2414fa5dd2d None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:33.089516 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c52fb-7ef0-4c4c-b119-f2414fa5dd2d None admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:33.090245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c52fb-7ef0-4c4c-b119-f2414fa5dd2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:33.090544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c52fb-7ef0-4c4c-b119-f2414fa5dd2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:33.091928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c52fb-7ef0-4c4c-b119-f2414fa5dd2d None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:33.099330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 637/1264] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:16:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:33.105585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71da988-43e7-4d2c-ba30-082eb666d611 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:33.208687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71da988-43e7-4d2c-ba30-082eb666d611 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:33.240003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfab0d-3796-4b7a-b0e0-a75eb5cd48a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:33.240790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfab0d-3796-4b7a-b0e0-a75eb5cd48a3 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:33.241072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfab0d-3796-4b7a-b0e0-a75eb5cd48a3 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:33.241343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfab0d-3796-4b7a-b0e0-a75eb5cd48a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:33.251187 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfab0d-3796-4b7a-b0e0-a75eb5cd48a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:33.252643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfab0d-3796-4b7a-b0e0-a75eb5cd48a3 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-72cfab0d-3796-4b7a-b0e0-a75eb5cd48a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:16:33.268747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:33.269743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:33.276037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:16:33.276037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:16:33.276037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:33.276037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:33.276037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 628/1265] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:33.276384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c52fb-7ef0-4c4c-b119-f2414fa5dd2d None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:33.370163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c52fb-7ef0-4c4c-b119-f2414fa5dd2d None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:33.400403 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b83496-2a87-48ab-acd7-08a5d4f0d1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:33.401079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b83496-2a87-48ab-acd7-08a5d4f0d1f3 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:33.401359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b83496-2a87-48ab-acd7-08a5d4f0d1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:33.401630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b83496-2a87-48ab-acd7-08a5d4f0d1f3 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:33.402339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b83496-2a87-48ab-acd7-08a5d4f0d1f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:33.402638 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b83496-2a87-48ab-acd7-08a5d4f0d1f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:33.404006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b83496-2a87-48ab-acd7-08a5d4f0d1f3 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:33.414851 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 638/1266] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:33.420879 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfab0d-3796-4b7a-b0e0-a75eb5cd48a3 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:33.513540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfab0d-3796-4b7a-b0e0-a75eb5cd48a3 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:33.544750 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bf61d-c20e-4c0d-9e74-17d115d1c440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:33.545537 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bf61d-c20e-4c0d-9e74-17d115d1c440 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:33.545823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bf61d-c20e-4c0d-9e74-17d115d1c440 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:33.546102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bf61d-c20e-4c0d-9e74-17d115d1c440 None admin] PATH_INFO: `/v3/users/placement` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:33.553240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bf61d-c20e-4c0d-9e74-17d115d1c440 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:33.554657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bf61d-c20e-4c0d-9e74-17d115d1c440 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-df0bf61d-c20e-4c0d-9e74-17d115d1c440 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:16:33.569137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:33.570218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:33.571411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:33.571411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:33.571411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:16:33.571411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:16:33.571411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:16:33.571411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:16:33.571411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:16:33.571411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:16:33.571411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 04:16:33.571411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:33.571411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 629/1267] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 04:16:33.575917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b83496-2a87-48ab-acd7-08a5d4f0d1f3 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:33.667574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b83496-2a87-48ab-acd7-08a5d4f0d1f3 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:33.699019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c77ba-fa0c-4197-b633-96e0f96bf6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:33.699667 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c77ba-fa0c-4197-b633-96e0f96bf6cf None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:33.699983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c77ba-fa0c-4197-b633-96e0f96bf6cf None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:33.700258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c77ba-fa0c-4197-b633-96e0f96bf6cf None admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:33.701082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c77ba-fa0c-4197-b633-96e0f96bf6cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:33.701385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c77ba-fa0c-4197-b633-96e0f96bf6cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:33.702796 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c77ba-fa0c-4197-b633-96e0f96bf6cf None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:33.719589 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 639/1268] 10.222.0.44 () {56 vars in 1143 bytes} [Mon May 9 04:16:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:33.724988 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bf61d-c20e-4c0d-9e74-17d115d1c440 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:33.822820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bf61d-c20e-4c0d-9e74-17d115d1c440 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:33.855355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c33d55-20ad-4c82-96d3-d23d3135da1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:33.856169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c33d55-20ad-4c82-96d3-d23d3135da1f None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:33.856456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c33d55-20ad-4c82-96d3-d23d3135da1f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:33.856733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c33d55-20ad-4c82-96d3-d23d3135da1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:33.866708 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c33d55-20ad-4c82-96d3-d23d3135da1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:33.868168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c33d55-20ad-4c82-96d3-d23d3135da1f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-89c33d55-20ad-4c82-96d3-d23d3135da1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:16:33.884760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:33.885861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:33.887128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:16:33.887128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:16:33.887128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:33.887128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:33.887128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 630/1269] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:33.891639 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c77ba-fa0c-4197-b633-96e0f96bf6cf None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:33.984628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c77ba-fa0c-4197-b633-96e0f96bf6cf None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:34.017034 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cddc2-2c73-4342-8573-de7b589ca0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:34.017680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cddc2-2c73-4342-8573-de7b589ca0c5 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:34.017960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cddc2-2c73-4342-8573-de7b589ca0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:34.018232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cddc2-2c73-4342-8573-de7b589ca0c5 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:34.018938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cddc2-2c73-4342-8573-de7b589ca0c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:34.019239 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cddc2-2c73-4342-8573-de7b589ca0c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:34.020615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cddc2-2c73-4342-8573-de7b589ca0c5 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:34.031185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 640/1270] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:34.036602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c33d55-20ad-4c82-96d3-d23d3135da1f None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:34.128362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c33d55-20ad-4c82-96d3-d23d3135da1f None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:34.159256 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb84e8-ca0f-460f-823f-31caa539a664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:34.160050 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb84e8-ca0f-460f-823f-31caa539a664 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:34.160331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb84e8-ca0f-460f-823f-31caa539a664 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:34.160602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb84e8-ca0f-460f-823f-31caa539a664 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:34.167403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb84e8-ca0f-460f-823f-31caa539a664 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:34.168852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb84e8-ca0f-460f-823f-31caa539a664 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-04eb84e8-ca0f-460f-823f-31caa539a664 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:16:34.183448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:34.184517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:34.185778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:16:34.185778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:16:34.185778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:16:34.185778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:16:34.185778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:16:34.185778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:16:34.185778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:16:34.185778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:34.185778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 631/1271] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:16:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:34.190181 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cddc2-2c73-4342-8573-de7b589ca0c5 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:34.283229 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cddc2-2c73-4342-8573-de7b589ca0c5 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:34.314382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2443ea9-3d62-44ca-b288-2026eccf1b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:34.315031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2443ea9-3d62-44ca-b288-2026eccf1b58 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:34.315318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2443ea9-3d62-44ca-b288-2026eccf1b58 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:34.315592 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2443ea9-3d62-44ca-b288-2026eccf1b58 None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:34.316396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2443ea9-3d62-44ca-b288-2026eccf1b58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:34.316704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2443ea9-3d62-44ca-b288-2026eccf1b58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:34.318117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2443ea9-3d62-44ca-b288-2026eccf1b58 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:34.329978 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 641/1272] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:16:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:34.336005 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb84e8-ca0f-460f-823f-31caa539a664 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:34.428272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb84e8-ca0f-460f-823f-31caa539a664 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:34.462356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e414806-5150-4f29-ba40-a4456c352c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:34.463968 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e414806-5150-4f29-ba40-a4456c352c94 None admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:34.464252 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e414806-5150-4f29-ba40-a4456c352c94 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:34.464525 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e414806-5150-4f29-ba40-a4456c352c94 None admin] PATH_INFO: `/v3/projects/3322b00723974bd99e837d447bac22d1/users/d41e05b7ef954dd596c0134b65434dd6/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:34.466405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e414806-5150-4f29-ba40-a4456c352c94 None admin] RBAC: Authorizing `identity:create_grant(project_id=3322b00723974bd99e837d447bac22d1, user_id=d41e05b7ef954dd596c0134b65434dd6, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:34.467808 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e414806-5150-4f29-ba40-a4456c352c94 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:34.481285 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 632/1273] 10.222.0.44 () {58 vars in 1439 bytes} [Mon May 9 04:16:34 2022] PUT /identity/v3/projects/3322b00723974bd99e837d447bac22d1/users/d41e05b7ef954dd596c0134b65434dd6/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:16:36.658453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10f8bc-ec30-47e8-9b6a-7b98a1c1a96f None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:36.659020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10f8bc-ec30-47e8-9b6a-7b98a1c1a96f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:36.659020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10f8bc-ec30-47e8-9b6a-7b98a1c1a96f None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:36.660407 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 642/1274] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:16:36.665541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646e019-d4a5-4e9d-bdb2-1f8a96ffa3f1 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:36.666022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646e019-d4a5-4e9d-bdb2-1f8a96ffa3f1 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:36.666304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646e019-d4a5-4e9d-bdb2-1f8a96ffa3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:36.682274 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-b646e019-d4a5-4e9d-bdb2-1f8a96ffa3f1 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:16:36.683432 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b646e019-d4a5-4e9d-bdb2-1f8a96ffa3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:16:36.751430 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 633/1275] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:16:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:16:36.776289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10f8bc-ec30-47e8-9b6a-7b98a1c1a96f None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:36.873311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10f8bc-ec30-47e8-9b6a-7b98a1c1a96f None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:36.905230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8700bb7-6a58-4b53-8bc1-1555c73d5136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:36.906016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8700bb7-6a58-4b53-8bc1-1555c73d5136 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:36.906296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8700bb7-6a58-4b53-8bc1-1555c73d5136 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:36.906568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8700bb7-6a58-4b53-8bc1-1555c73d5136 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:36.916741 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8700bb7-6a58-4b53-8bc1-1555c73d5136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-a8700bb7-6a58-4b53-8bc1-1555c73d5136 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 04:16:36.924220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:36.925282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:36.926318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:36.926318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:36.926318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:36.926318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:36.926318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 04:16:36.926318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 04:16:36.926318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 04:16:36.926318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 04:16:36.926318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 04:16:36.926318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 04:16:36.926318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 04:16:36.926318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:36.926318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 643/1276] 10.222.0.44 () {56 vars in 1096 bytes} [Mon May 9 04:16:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:36.931850 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646e019-d4a5-4e9d-bdb2-1f8a96ffa3f1 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:37.024333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646e019-d4a5-4e9d-bdb2-1f8a96ffa3f1 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:37.057343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d573fd-e4a4-45a3-9ba4-b6341630e9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:37.057991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d573fd-e4a4-45a3-9ba4-b6341630e9bc None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:37.058268 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d573fd-e4a4-45a3-9ba4-b6341630e9bc None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:37.058540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d573fd-e4a4-45a3-9ba4-b6341630e9bc None admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:37.059274 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d573fd-e4a4-45a3-9ba4-b6341630e9bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:37.059574 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d573fd-e4a4-45a3-9ba4-b6341630e9bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:37.060955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d573fd-e4a4-45a3-9ba4-b6341630e9bc None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:37.068449 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 634/1277] 10.222.0.44 () {56 vars in 1099 bytes} [Mon May 9 04:16:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:37.074890 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8700bb7-6a58-4b53-8bc1-1555c73d5136 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:37.170141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8700bb7-6a58-4b53-8bc1-1555c73d5136 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:37.200712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff3880-50da-47f5-9d1a-367e6201e3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:37.201489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff3880-50da-47f5-9d1a-367e6201e3bb None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:37.201777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff3880-50da-47f5-9d1a-367e6201e3bb None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:37.202049 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff3880-50da-47f5-9d1a-367e6201e3bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:37.212100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff3880-50da-47f5-9d1a-367e6201e3bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:37.214580 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff3880-50da-47f5-9d1a-367e6201e3bb None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-52ff3880-50da-47f5-9d1a-367e6201e3bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:16:37.230944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:37.232043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:37.233175 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:16:37.233175 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:16:37.233175 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:37.233175 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:37.233175 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 644/1278] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:37.238001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d573fd-e4a4-45a3-9ba4-b6341630e9bc None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:37.330907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d573fd-e4a4-45a3-9ba4-b6341630e9bc None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:37.361547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67bfba-212e-4831-95bb-5659563f0b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:37.362193 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67bfba-212e-4831-95bb-5659563f0b41 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:37.362472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67bfba-212e-4831-95bb-5659563f0b41 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:37.362742 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67bfba-212e-4831-95bb-5659563f0b41 None admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:37.363447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67bfba-212e-4831-95bb-5659563f0b41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:37.363747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67bfba-212e-4831-95bb-5659563f0b41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:37.365111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67bfba-212e-4831-95bb-5659563f0b41 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:37.376136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 635/1279] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:37.381792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff3880-50da-47f5-9d1a-367e6201e3bb None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:37.474914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff3880-50da-47f5-9d1a-367e6201e3bb None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:37.506353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fd150-de77-4b52-9a99-9fc03a41be68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:37.507147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fd150-de77-4b52-9a99-9fc03a41be68 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:37.507429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fd150-de77-4b52-9a99-9fc03a41be68 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:37.507703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fd150-de77-4b52-9a99-9fc03a41be68 None admin] PATH_INFO: `/v3/users/placement` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:37.515396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fd150-de77-4b52-9a99-9fc03a41be68 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:37.516852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fd150-de77-4b52-9a99-9fc03a41be68 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-247fd150-de77-4b52-9a99-9fc03a41be68 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 04:16:37.530635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:37.531697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:37.532961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:37.532961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:37.532961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 04:16:37.532961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 04:16:37.532961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 04:16:37.532961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 04:16:37.532961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 04:16:37.532961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 04:16:37.532961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 04:16:37.532961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:37.532961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 645/1280] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:37.537525 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67bfba-212e-4831-95bb-5659563f0b41 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:37.633257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67bfba-212e-4831-95bb-5659563f0b41 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:37.665263 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5da4f-c67e-4559-80ea-0695aa6af4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:37.666131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5da4f-c67e-4559-80ea-0695aa6af4e8 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:37.666685 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5da4f-c67e-4559-80ea-0695aa6af4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:37.666685 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5da4f-c67e-4559-80ea-0695aa6af4e8 None admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:37.667505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5da4f-c67e-4559-80ea-0695aa6af4e8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:37.667813 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5da4f-c67e-4559-80ea-0695aa6af4e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:37.669245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5da4f-c67e-4559-80ea-0695aa6af4e8 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:37.686299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 636/1281] 10.222.0.44 () {56 vars in 1143 bytes} [Mon May 9 04:16:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:37.692001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fd150-de77-4b52-9a99-9fc03a41be68 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:37.789900 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fd150-de77-4b52-9a99-9fc03a41be68 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:37.821686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc03426-64d7-43cd-b1f7-b6a7d9219cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:37.822962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc03426-64d7-43cd-b1f7-b6a7d9219cb2 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:37.823239 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc03426-64d7-43cd-b1f7-b6a7d9219cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:37.823508 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc03426-64d7-43cd-b1f7-b6a7d9219cb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:37.884939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc03426-64d7-43cd-b1f7-b6a7d9219cb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:37.886365 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc03426-64d7-43cd-b1f7-b6a7d9219cb2 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:37.980455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 646/1282] 10.222.0.44 () {58 vars in 1282 bytes} [Mon May 9 04:16:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 04:16:37.986193 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5da4f-c67e-4559-80ea-0695aa6af4e8 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:38.077681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5da4f-c67e-4559-80ea-0695aa6af4e8 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:38.108760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e22a9d-b2d5-4fd1-a9a2-65d62cbb3c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:38.109527 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e22a9d-b2d5-4fd1-a9a2-65d62cbb3c42 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:38.109805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e22a9d-b2d5-4fd1-a9a2-65d62cbb3c42 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:38.110078 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e22a9d-b2d5-4fd1-a9a2-65d62cbb3c42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:38.120009 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e22a9d-b2d5-4fd1-a9a2-65d62cbb3c42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:38.121414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e22a9d-b2d5-4fd1-a9a2-65d62cbb3c42 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-48e22a9d-b2d5-4fd1-a9a2-65d62cbb3c42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 04:16:38.138017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:38.139083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:38.140261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 04:16:38.140261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 04:16:38.140261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 04:16:38.140261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:38.140261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 637/1283] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:38.144744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc03426-64d7-43cd-b1f7-b6a7d9219cb2 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:38.237185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc03426-64d7-43cd-b1f7-b6a7d9219cb2 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:38.267842 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f7b8a-acb1-46b5-ad43-511bcd024095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:38.268520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f7b8a-acb1-46b5-ad43-511bcd024095 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:38.268800 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f7b8a-acb1-46b5-ad43-511bcd024095 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:38.269071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f7b8a-acb1-46b5-ad43-511bcd024095 None admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:38.269777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f7b8a-acb1-46b5-ad43-511bcd024095 None admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:38.270079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f7b8a-acb1-46b5-ad43-511bcd024095 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:38.271428 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f7b8a-acb1-46b5-ad43-511bcd024095 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:38.282497 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 647/1284] 10.222.0.44 () {56 vars in 1109 bytes} [Mon May 9 04:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:38.288151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e22a9d-b2d5-4fd1-a9a2-65d62cbb3c42 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:38.380784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e22a9d-b2d5-4fd1-a9a2-65d62cbb3c42 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:38.411983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf89df-8645-4d9b-904c-58bd5af1cab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:38.412748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf89df-8645-4d9b-904c-58bd5af1cab6 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:38.413026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf89df-8645-4d9b-904c-58bd5af1cab6 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:38.413296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf89df-8645-4d9b-904c-58bd5af1cab6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:38.420200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf89df-8645-4d9b-904c-58bd5af1cab6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:38.421623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf89df-8645-4d9b-904c-58bd5af1cab6 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application [None req-8aaf89df-8645-4d9b-904c-58bd5af1cab6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 04:16:38.434713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application with Lock( May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:38.435732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:38.436783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 04:16:38.436783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 04:16:38.436783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 04:16:38.436783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 04:16:38.436783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 04:16:38.436783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 04:16:38.436783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 04:16:38.436783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: ERROR keystone.server.flask.application May 09 04:16:38.436783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 638/1285] 10.222.0.44 () {56 vars in 1111 bytes} [Mon May 9 04:16:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:38.441443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f7b8a-acb1-46b5-ad43-511bcd024095 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:38.534049 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f7b8a-acb1-46b5-ad43-511bcd024095 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:38.564990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b56f1-ad42-424c-ad28-c614a7d7a78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:38.565625 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b56f1-ad42-424c-ad28-c614a7d7a78a None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:38.565902 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b56f1-ad42-424c-ad28-c614a7d7a78a None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:38.566171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b56f1-ad42-424c-ad28-c614a7d7a78a None admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:38.566939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b56f1-ad42-424c-ad28-c614a7d7a78a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:38.567237 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b56f1-ad42-424c-ad28-c614a7d7a78a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:38.568662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b56f1-ad42-424c-ad28-c614a7d7a78a None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:38.580715 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 648/1286] 10.222.0.44 () {56 vars in 1148 bytes} [Mon May 9 04:16:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:38.586588 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf89df-8645-4d9b-904c-58bd5af1cab6 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:38.679783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf89df-8645-4d9b-904c-58bd5af1cab6 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:38.710979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11583b97-6500-46b2-b8b4-20b72e98e869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:38.711616 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11583b97-6500-46b2-b8b4-20b72e98e869 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:38.711916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11583b97-6500-46b2-b8b4-20b72e98e869 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:38.712200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11583b97-6500-46b2-b8b4-20b72e98e869 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:38.713057 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11583b97-6500-46b2-b8b4-20b72e98e869 None admin] RBAC: Adding query filter params (role.id=e59113c0366049e889df3f55eed90c72, scope.project.id=3322b00723974bd99e837d447bac22d1, user.id=d41e05b7ef954dd596c0134b65434dd6) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:38.713365 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11583b97-6500-46b2-b8b4-20b72e98e869 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:38.714769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11583b97-6500-46b2-b8b4-20b72e98e869 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:38.720302 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 639/1287] 10.222.0.44 () {56 vars in 1374 bytes} [Mon May 9 04:16:38 2022] GET /identity/v3/role_assignments?user.id=d41e05b7ef954dd596c0134b65434dd6&scope.project.id=3322b00723974bd99e837d447bac22d1&role.id=e59113c0366049e889df3f55eed90c72 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:40.933959 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd23d14-c027-4e56-882d-f4f777f2f88b None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:40.934479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd23d14-c027-4e56-882d-f4f777f2f88b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:40.934479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd23d14-c027-4e56-882d-f4f777f2f88b None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:40.935868 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 649/1288] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:16:40.941116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5d2b4-6dcf-4fb9-a522-61ed82201fec None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:40.941513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5d2b4-6dcf-4fb9-a522-61ed82201fec None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:40.941674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5d2b4-6dcf-4fb9-a522-61ed82201fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:40.958505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-27b5d2b4-6dcf-4fb9-a522-61ed82201fec None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:16:40.959798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-27b5d2b4-6dcf-4fb9-a522-61ed82201fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:16:41.021796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 640/1289] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:16:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:16:41.043932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd23d14-c027-4e56-882d-f4f777f2f88b None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:41.138519 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd23d14-c027-4e56-882d-f4f777f2f88b None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:41.169548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7d889-b56e-4546-a502-0dc98f43bea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:41.170323 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7d889-b56e-4546-a502-0dc98f43bea5 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:41.170599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7d889-b56e-4546-a502-0dc98f43bea5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:41.170870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7d889-b56e-4546-a502-0dc98f43bea5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:41.171515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec7d889-b56e-4546-a502-0dc98f43bea5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:41.172925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec7d889-b56e-4546-a502-0dc98f43bea5 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-2ec7d889-b56e-4546-a502-0dc98f43bea5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:41.184714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:41.185927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:41.186953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 650/1290] 10.222.0.44 () {56 vars in 1117 bytes} [Mon May 9 04:16:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:41.191864 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5d2b4-6dcf-4fb9-a522-61ed82201fec None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:41.284520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5d2b4-6dcf-4fb9-a522-61ed82201fec None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:41.316401 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605aab7-d9f1-4346-83e0-671adb7571f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:41.317048 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605aab7-d9f1-4346-83e0-671adb7571f1 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:41.317333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605aab7-d9f1-4346-83e0-671adb7571f1 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:41.317611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605aab7-d9f1-4346-83e0-671adb7571f1 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:41.318311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605aab7-d9f1-4346-83e0-671adb7571f1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:41.318617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605aab7-d9f1-4346-83e0-671adb7571f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:41.319990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605aab7-d9f1-4346-83e0-671adb7571f1 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:41.324815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 641/1291] 10.222.0.44 () {56 vars in 1116 bytes} [Mon May 9 04:16:41 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:41.330681 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7d889-b56e-4546-a502-0dc98f43bea5 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:41.421970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7d889-b56e-4546-a502-0dc98f43bea5 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:41.452514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab38e3-c6bc-449c-b9dc-22a07f9fb70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:41.453170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab38e3-c6bc-449c-b9dc-22a07f9fb70f None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:41.453454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab38e3-c6bc-449c-b9dc-22a07f9fb70f None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:41.453730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab38e3-c6bc-449c-b9dc-22a07f9fb70f None admin] PATH_INFO: `/v3/services` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:41.454432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab38e3-c6bc-449c-b9dc-22a07f9fb70f None admin] RBAC: Adding query filter params (type=placement) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:41.454738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab38e3-c6bc-449c-b9dc-22a07f9fb70f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:41.456378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab38e3-c6bc-449c-b9dc-22a07f9fb70f None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:41.460619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 651/1292] 10.222.0.44 () {56 vars in 1116 bytes} [Mon May 9 04:16:41 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:43.666691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff4d85-fe9b-4e32-916a-92379006a0c6 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:43.667254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff4d85-fe9b-4e32-916a-92379006a0c6 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:43.667254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff4d85-fe9b-4e32-916a-92379006a0c6 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:43.668628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 642/1293] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:16:43.673800 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da9a05-00ab-48b3-ac70-ac668a31448e None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:43.674233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da9a05-00ab-48b3-ac70-ac668a31448e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:43.674349 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da9a05-00ab-48b3-ac70-ac668a31448e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:43.690312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-a8da9a05-00ab-48b3-ac70-ac668a31448e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:16:43.691462 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a8da9a05-00ab-48b3-ac70-ac668a31448e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:16:43.753057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 652/1294] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:16:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:16:43.777831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff4d85-fe9b-4e32-916a-92379006a0c6 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:43.882053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff4d85-fe9b-4e32-916a-92379006a0c6 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:43.913800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26888551-803b-4ba5-9b83-d816465c997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:43.914451 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26888551-803b-4ba5-9b83-d816465c997f None admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:43.914735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26888551-803b-4ba5-9b83-d816465c997f None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:43.915009 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26888551-803b-4ba5-9b83-d816465c997f None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:43.916277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26888551-803b-4ba5-9b83-d816465c997f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:43.917669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26888551-803b-4ba5-9b83-d816465c997f None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:43.929479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 643/1295] 10.222.0.44 () {60 vars in 1141 bytes} [Mon May 9 04:16:43 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:16:46.154880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19fe89-2606-4f75-9a72-517c32b56c6c None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:46.155428 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19fe89-2606-4f75-9a72-517c32b56c6c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:46.155428 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19fe89-2606-4f75-9a72-517c32b56c6c None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:46.156739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 653/1296] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:16:46.162023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29954b52-64d5-4f61-8c24-b7f857f4308c None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:46.162439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29954b52-64d5-4f61-8c24-b7f857f4308c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:46.162563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29954b52-64d5-4f61-8c24-b7f857f4308c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:46.179161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-29954b52-64d5-4f61-8c24-b7f857f4308c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:16:46.180347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-29954b52-64d5-4f61-8c24-b7f857f4308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:16:46.248143 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 644/1297] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:16:46 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:16:46.270270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19fe89-2606-4f75-9a72-517c32b56c6c None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:46.367465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19fe89-2606-4f75-9a72-517c32b56c6c None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:46.399710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8144c-1372-40eb-94fe-0d85a846a9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:46.401085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8144c-1372-40eb-94fe-0d85a846a9f5 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:46.401373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8144c-1372-40eb-94fe-0d85a846a9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:46.401652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8144c-1372-40eb-94fe-0d85a846a9f5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:46.402308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8144c-1372-40eb-94fe-0d85a846a9f5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:46.403668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8144c-1372-40eb-94fe-0d85a846a9f5 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-f9a8144c-1372-40eb-94fe-0d85a846a9f5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:46.415107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:46.416784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:46.417827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 654/1298] 10.222.0.44 () {56 vars in 1117 bytes} [Mon May 9 04:16:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:46.423357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29954b52-64d5-4f61-8c24-b7f857f4308c None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:46.518531 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29954b52-64d5-4f61-8c24-b7f857f4308c None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:46.551546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a8658-2ab2-4bc1-9765-611c84ee4e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:46.552349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a8658-2ab2-4bc1-9765-611c84ee4e88 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:46.552633 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a8658-2ab2-4bc1-9765-611c84ee4e88 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:46.552910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a8658-2ab2-4bc1-9765-611c84ee4e88 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:46.553610 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a8658-2ab2-4bc1-9765-611c84ee4e88 None admin] RBAC: Adding query filter params (name=placement) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:46.553918 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a8658-2ab2-4bc1-9765-611c84ee4e88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:46.555267 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a8658-2ab2-4bc1-9765-611c84ee4e88 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:46.563689 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 645/1299] 10.222.0.44 () {56 vars in 1116 bytes} [Mon May 9 04:16:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:46.569490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8144c-1372-40eb-94fe-0d85a846a9f5 None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:46.665984 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8144c-1372-40eb-94fe-0d85a846a9f5 None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:46.696245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f549c-d1ec-4b30-9a03-bc8885e782e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:46.696905 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f549c-d1ec-4b30-9a03-bc8885e782e5 None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:46.697264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f549c-d1ec-4b30-9a03-bc8885e782e5 None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:46.697535 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f549c-d1ec-4b30-9a03-bc8885e782e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:46.698305 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f549c-d1ec-4b30-9a03-bc8885e782e5 None admin] RBAC: Adding query filter params (interface=public, service_id=ddc8f7f59bf84b3b8b8d7d017863c129, region_id=RegionOne) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:46.698609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f549c-d1ec-4b30-9a03-bc8885e782e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:46.699983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f549c-d1ec-4b30-9a03-bc8885e782e5 None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:46.704669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 655/1300] 10.222.0.44 () {56 vars in 1251 bytes} [Mon May 9 04:16:46 2022] GET /identity/v3/endpoints?service_id=ddc8f7f59bf84b3b8b8d7d017863c129&interface=public®ion_id=RegionOne => generated 194 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:46.710520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a8658-2ab2-4bc1-9765-611c84ee4e88 None admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:46.805006 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a8658-2ab2-4bc1-9765-611c84ee4e88 None admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:46.836450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbac6d-ced0-4a3d-93ea-efebae970622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:46.837094 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbac6d-ced0-4a3d-93ea-efebae970622 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:46.837945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbac6d-ced0-4a3d-93ea-efebae970622 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:46.838304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbac6d-ced0-4a3d-93ea-efebae970622 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:46.838893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbac6d-ced0-4a3d-93ea-efebae970622 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:46.840261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbac6d-ced0-4a3d-93ea-efebae970622 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:46.846691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 646/1301] 10.222.0.44 () {56 vars in 1087 bytes} [Mon May 9 04:16:46 2022] GET /identity/v3/services => generated 2191 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:16:49.161214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a86c1-a141-46d3-bf2b-f6d9d6dd8175 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:49.161715 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a86c1-a141-46d3-bf2b-f6d9d6dd8175 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:49.161715 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a86c1-a141-46d3-bf2b-f6d9d6dd8175 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:49.163056 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 656/1302] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:16:49.168188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec18e9e-39aa-420c-9fb6-963af8dfc92d None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:49.168606 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec18e9e-39aa-420c-9fb6-963af8dfc92d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:49.168740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec18e9e-39aa-420c-9fb6-963af8dfc92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:49.184681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-9ec18e9e-39aa-420c-9fb6-963af8dfc92d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:16:49.185832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9ec18e9e-39aa-420c-9fb6-963af8dfc92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:16:49.247505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 647/1303] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:16:49 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 04:16:49.270356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a86c1-a141-46d3-bf2b-f6d9d6dd8175 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:49.364234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a86c1-a141-46d3-bf2b-f6d9d6dd8175 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:49.394877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015727a8-4fce-4e47-8fb8-40fd2b35e3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:49.395658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015727a8-4fce-4e47-8fb8-40fd2b35e3cf None admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:49.395973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015727a8-4fce-4e47-8fb8-40fd2b35e3cf None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:49.396246 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015727a8-4fce-4e47-8fb8-40fd2b35e3cf None admin] PATH_INFO: `/v3/services/placement` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:49.396887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015727a8-4fce-4e47-8fb8-40fd2b35e3cf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:49.398241 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015727a8-4fce-4e47-8fb8-40fd2b35e3cf None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application [None req-015727a8-4fce-4e47-8fb8-40fd2b35e3cf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 04:16:49.410311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 04:16:49.411485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.get_or_create( May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application with Lock( May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self._enter() May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application return self.creator() May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application created_value = creator( May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: ERROR keystone.server.flask.application May 09 04:16:49.412548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 657/1304] 10.222.0.44 () {56 vars in 1117 bytes} [Mon May 9 04:16:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 04:16:49.417434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec18e9e-39aa-420c-9fb6-963af8dfc92d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:49.508406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec18e9e-39aa-420c-9fb6-963af8dfc92d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:49.539612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2fdb9-47f2-4034-acb8-3d41c6e43498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:49.540311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2fdb9-47f2-4034-acb8-3d41c6e43498 None admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:49.540595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2fdb9-47f2-4034-acb8-3d41c6e43498 None admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:49.541039 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2fdb9-47f2-4034-acb8-3d41c6e43498 None admin] PATH_INFO: `/v3/services` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:49.541735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2fdb9-47f2-4034-acb8-3d41c6e43498 None admin] RBAC: Adding query filter params (name=placement) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:16:49.542038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2fdb9-47f2-4034-acb8-3d41c6e43498 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:49.543373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2fdb9-47f2-4034-acb8-3d41c6e43498 None admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:49.548228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 648/1305] 10.222.0.44 () {56 vars in 1116 bytes} [Mon May 9 04:16:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:16:49.553769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015727a8-4fce-4e47-8fb8-40fd2b35e3cf None admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:16:49.646929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015727a8-4fce-4e47-8fb8-40fd2b35e3cf None admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:16:49.677348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf4c26-a443-44eb-b7de-9209cbe119ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:16:49.677931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf4c26-a443-44eb-b7de-9209cbe119ef None admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:16:49.678211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf4c26-a443-44eb-b7de-9209cbe119ef None admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:16:49.678482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf4c26-a443-44eb-b7de-9209cbe119ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:16:49.679214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf4c26-a443-44eb-b7de-9209cbe119ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:16:49.680585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf4c26-a443-44eb-b7de-9209cbe119ef None admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:16:49.698654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 658/1306] 10.222.0.44 () {60 vars in 1144 bytes} [Mon May 9 04:16:49 2022] POST /identity/v3/endpoints => generated 331 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:17:53.533455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e124d-2130-4cd7-8ba6-aa774be0b053 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:17:53.534499 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e124d-2130-4cd7-8ba6-aa774be0b053 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:17:53.534999 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e124d-2130-4cd7-8ba6-aa774be0b053 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:17:53.542818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d6f08-0780-4f2c-bf66-c5d206de6c1b None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:17:53.543700 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d6f08-0780-4f2c-bf66-c5d206de6c1b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:17:53.544232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d6f08-0780-4f2c-bf66-c5d206de6c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:17:53.547770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 649/1307] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:17:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:17:53.618587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-092d6f08-0780-4f2c-bf66-c5d206de6c1b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:17:53.620284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-092d6f08-0780-4f2c-bf66-c5d206de6c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:17:53.743210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 659/1308] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:17:53.785549 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8dfe5-b974-49a2-9ce0-4d1f2a64a306 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:17:53.785912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8dfe5-b974-49a2-9ce0-4d1f2a64a306 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:17:53.786428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8dfe5-b974-49a2-9ce0-4d1f2a64a306 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:17:53.788608 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 650/1309] 10.222.0.44 () {54 vars in 964 bytes} [Mon May 9 04:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:17:53.797768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73509c9-7318-44a1-bcbf-026e722cc742 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:17:53.798088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73509c9-7318-44a1-bcbf-026e722cc742 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:17:53.798581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73509c9-7318-44a1-bcbf-026e722cc742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:17:53.892428 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-f73509c9-7318-44a1-bcbf-026e722cc742 None None] MFA Rules not processed for user `0d852975a2e84925a45775a16cc52f32`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:17:53.893840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f73509c9-7318-44a1-bcbf-026e722cc742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:17:53.999353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8dfe5-b974-49a2-9ce0-4d1f2a64a306 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:17:54.001382 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-00a8dfe5-b974-49a2-9ce0-4d1f2a64a306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:17:54.005643 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 660/1310] 10.222.0.44 () {58 vars in 1061 bytes} [Mon May 9 04:17:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:17:54.155558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8dfe5-b974-49a2-9ce0-4d1f2a64a306 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:17:54.232051 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2e23d-adeb-4e4b-bacf-4e90d7690421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:17:54.232715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2e23d-adeb-4e4b-bacf-4e90d7690421 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:17:54.233002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2e23d-adeb-4e4b-bacf-4e90d7690421 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:17:54.233281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2e23d-adeb-4e4b-bacf-4e90d7690421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:17:54.234913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-90a2e23d-adeb-4e4b-bacf-4e90d7690421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:17:54.352678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2e23d-adeb-4e4b-bacf-4e90d7690421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:17:54.354961 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2e23d-adeb-4e4b-bacf-4e90d7690421 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:17:54.379384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 651/1311] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:17:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:17:56.940272 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da76dd-9045-4cbe-a5d8-9f0416379392 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:17:56.940850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da76dd-9045-4cbe-a5d8-9f0416379392 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:17:56.940850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da76dd-9045-4cbe-a5d8-9f0416379392 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:17:56.943367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 661/1312] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:17:56.947386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13f879-1378-4d9f-b3dc-061bade9f1c8 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:17:56.947828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13f879-1378-4d9f-b3dc-061bade9f1c8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:17:56.947946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13f879-1378-4d9f-b3dc-061bade9f1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:17:57.030002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-1e13f879-1378-4d9f-b3dc-061bade9f1c8 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:17:57.031240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1e13f879-1378-4d9f-b3dc-061bade9f1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:17:57.129172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 652/1313] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:17:57.167455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da76dd-9045-4cbe-a5d8-9f0416379392 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:17:57.169223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-26da76dd-9045-4cbe-a5d8-9f0416379392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:17:57.272600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da76dd-9045-4cbe-a5d8-9f0416379392 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:17:57.346175 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c11f8-e493-402e-b9b5-066b74b99563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:17:57.346831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c11f8-e493-402e-b9b5-066b74b99563 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:17:57.347111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c11f8-e493-402e-b9b5-066b74b99563 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:17:57.347438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c11f8-e493-402e-b9b5-066b74b99563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:17:57.349250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-dd4c11f8-e493-402e-b9b5-066b74b99563 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:17:57.457239 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c11f8-e493-402e-b9b5-066b74b99563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:17:57.458681 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c11f8-e493-402e-b9b5-066b74b99563 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:17:57.480094 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 662/1314] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:17:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:00.058843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025e786-0f42-4fff-bbbb-006f9e7db4d5 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:00.059378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025e786-0f42-4fff-bbbb-006f9e7db4d5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:00.059378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025e786-0f42-4fff-bbbb-006f9e7db4d5 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:00.060692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 653/1315] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:00.065963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560bad8-0655-4bb4-aa1c-b37343dabe5c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:00.066376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560bad8-0655-4bb4-aa1c-b37343dabe5c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:00.066523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560bad8-0655-4bb4-aa1c-b37343dabe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:00.129995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-1560bad8-0655-4bb4-aa1c-b37343dabe5c None None] MFA Rules not processed for user `3248cf2854434b85b81170d82133b435`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:00.131192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1560bad8-0655-4bb4-aa1c-b37343dabe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:00.223430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 663/1316] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:00.264338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025e786-0f42-4fff-bbbb-006f9e7db4d5 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:00.265977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0025e786-0f42-4fff-bbbb-006f9e7db4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:00.367589 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025e786-0f42-4fff-bbbb-006f9e7db4d5 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:00.436692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ef087-2dbb-4513-9274-017dbe8c89d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:00.437344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ef087-2dbb-4513-9274-017dbe8c89d1 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:00.437625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ef087-2dbb-4513-9274-017dbe8c89d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:00.437896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ef087-2dbb-4513-9274-017dbe8c89d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:00.439515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5f9ef087-2dbb-4513-9274-017dbe8c89d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:00.543699 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ef087-2dbb-4513-9274-017dbe8c89d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:00.545160 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ef087-2dbb-4513-9274-017dbe8c89d1 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:00.562764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 654/1317] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:03.387488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de9cf8-3f36-4140-9f2e-1ecc1f86d993 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:03.388079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de9cf8-3f36-4140-9f2e-1ecc1f86d993 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:03.388369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de9cf8-3f36-4140-9f2e-1ecc1f86d993 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:03.390494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 664/1318] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:03.397393 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a678d-53f0-457f-86fd-69c2efae782e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:03.397788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a678d-53f0-457f-86fd-69c2efae782e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:03.397993 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a678d-53f0-457f-86fd-69c2efae782e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:03.471718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f77a678d-53f0-457f-86fd-69c2efae782e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:03.472994 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f77a678d-53f0-457f-86fd-69c2efae782e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:03.582507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 655/1319] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:03.621752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de9cf8-3f36-4140-9f2e-1ecc1f86d993 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:03.623274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-43de9cf8-3f36-4140-9f2e-1ecc1f86d993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:03.720705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de9cf8-3f36-4140-9f2e-1ecc1f86d993 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:03.799686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de711969-b923-4ba1-9095-49bbeb48c09f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:03.800356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de711969-b923-4ba1-9095-49bbeb48c09f service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:03.800636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de711969-b923-4ba1-9095-49bbeb48c09f service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:03.800909 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de711969-b923-4ba1-9095-49bbeb48c09f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:03.802523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-de711969-b923-4ba1-9095-49bbeb48c09f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:03.905423 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de711969-b923-4ba1-9095-49bbeb48c09f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:03.906852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de711969-b923-4ba1-9095-49bbeb48c09f service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:03.925002 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 665/1320] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:06.781882 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b88ede-eb9e-429b-977c-13bc6b16d6b3 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:06.782400 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b88ede-eb9e-429b-977c-13bc6b16d6b3 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:06.782494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b88ede-eb9e-429b-977c-13bc6b16d6b3 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:06.783846 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 656/1321] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:06.789253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dc0e2-b03f-4fe4-bd88-fc05dbb37985 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:06.789703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dc0e2-b03f-4fe4-bd88-fc05dbb37985 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:06.789925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dc0e2-b03f-4fe4-bd88-fc05dbb37985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:06.853606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-5b6dc0e2-b03f-4fe4-bd88-fc05dbb37985 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:06.854795 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5b6dc0e2-b03f-4fe4-bd88-fc05dbb37985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:06.949573 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 666/1322] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:18:06.997942 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff133e3a-9218-440b-a685-2fc5f5ddfdef None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:06.998317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff133e3a-9218-440b-a685-2fc5f5ddfdef None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:06.998666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff133e3a-9218-440b-a685-2fc5f5ddfdef None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:07.000235 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 657/1323] 10.222.0.44 () {54 vars in 964 bytes} [Mon May 9 04:18:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:07.006552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0ba08-9ebe-4926-a54f-857bcdb539db None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:07.006919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0ba08-9ebe-4926-a54f-857bcdb539db None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:07.007267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0ba08-9ebe-4926-a54f-857bcdb539db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:07.088177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-3af0ba08-9ebe-4926-a54f-857bcdb539db None None] MFA Rules not processed for user `0d852975a2e84925a45775a16cc52f32`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:07.089280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3af0ba08-9ebe-4926-a54f-857bcdb539db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:07.161047 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 667/1324] 10.222.0.44 () {58 vars in 1061 bytes} [Mon May 9 04:18:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:07.170404 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff133e3a-9218-440b-a685-2fc5f5ddfdef None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:07.171953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-ff133e3a-9218-440b-a685-2fc5f5ddfdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:07.274375 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff133e3a-9218-440b-a685-2fc5f5ddfdef None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:07.353564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ff4e3-3f4c-45da-a5ca-e1c1d0b749c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:07.354158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ff4e3-3f4c-45da-a5ca-e1c1d0b749c2 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:07.354437 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ff4e3-3f4c-45da-a5ca-e1c1d0b749c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:07.354704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ff4e3-3f4c-45da-a5ca-e1c1d0b749c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:07.356353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-695ff4e3-3f4c-45da-a5ca-e1c1d0b749c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:07.464893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ff4e3-3f4c-45da-a5ca-e1c1d0b749c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:07.466356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ff4e3-3f4c-45da-a5ca-e1c1d0b749c2 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:07.483414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 658/1325] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:10.091497 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de7576-5cd7-4245-b3c5-735ceb72fe2a None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:10.092041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de7576-5cd7-4245-b3c5-735ceb72fe2a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:10.092041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de7576-5cd7-4245-b3c5-735ceb72fe2a None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:10.093394 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 668/1326] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:10.099010 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32aea3-d446-49f9-8721-1de9db8cf0b9 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:10.099543 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32aea3-d446-49f9-8721-1de9db8cf0b9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:10.099543 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32aea3-d446-49f9-8721-1de9db8cf0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:10.166311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-3e32aea3-d446-49f9-8721-1de9db8cf0b9 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:10.167485 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3e32aea3-d446-49f9-8721-1de9db8cf0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:10.259223 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 659/1327] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:10.293915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de7576-5cd7-4245-b3c5-735ceb72fe2a None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:10.295434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d2de7576-5cd7-4245-b3c5-735ceb72fe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:10.397850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de7576-5cd7-4245-b3c5-735ceb72fe2a None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:10.484045 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd636d-3a7f-4bdc-942a-a9ce1b587d08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:10.484697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd636d-3a7f-4bdc-942a-a9ce1b587d08 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:10.484979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd636d-3a7f-4bdc-942a-a9ce1b587d08 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:10.485260 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd636d-3a7f-4bdc-942a-a9ce1b587d08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:10.486860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-0bcd636d-3a7f-4bdc-942a-a9ce1b587d08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:10.590223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd636d-3a7f-4bdc-942a-a9ce1b587d08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:10.591669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd636d-3a7f-4bdc-942a-a9ce1b587d08 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:10.610448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 669/1328] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:12.904958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc868-0657-4ed8-bf4c-8f03a20e365f None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:12.905481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc868-0657-4ed8-bf4c-8f03a20e365f None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:12.905593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc868-0657-4ed8-bf4c-8f03a20e365f None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:12.906872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 660/1329] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:12.912088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6afec-f4a9-4eb9-8594-e4fd7f2cc813 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:12.912514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6afec-f4a9-4eb9-8594-e4fd7f2cc813 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:12.912740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6afec-f4a9-4eb9-8594-e4fd7f2cc813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:12.979871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-a6e6afec-f4a9-4eb9-8594-e4fd7f2cc813 None None] MFA Rules not processed for user `3248cf2854434b85b81170d82133b435`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:12.981110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a6e6afec-f4a9-4eb9-8594-e4fd7f2cc813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:13.081334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 670/1330] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:13.119319 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dc868-0657-4ed8-bf4c-8f03a20e365f None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:13.120929 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5f1dc868-0657-4ed8-bf4c-8f03a20e365f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:13.219811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dc868-0657-4ed8-bf4c-8f03a20e365f None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:13.291006 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea4340-c065-4813-8e07-b1c8757c698a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:13.291656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea4340-c065-4813-8e07-b1c8757c698a service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:13.291975 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea4340-c065-4813-8e07-b1c8757c698a service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:13.292255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea4340-c065-4813-8e07-b1c8757c698a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:13.293884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-beea4340-c065-4813-8e07-b1c8757c698a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:13.398806 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea4340-c065-4813-8e07-b1c8757c698a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:13.400298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea4340-c065-4813-8e07-b1c8757c698a service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:13.417784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 661/1331] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:16.017460 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7159bc-3623-4d06-9c37-ca6fd477e3b5 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:16.017979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7159bc-3623-4d06-9c37-ca6fd477e3b5 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:16.017979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7159bc-3623-4d06-9c37-ca6fd477e3b5 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:16.019381 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 671/1332] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:16.024926 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc0347-8abf-451c-820a-00d4c346e9d9 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:16.025357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc0347-8abf-451c-820a-00d4c346e9d9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:16.025489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc0347-8abf-451c-820a-00d4c346e9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:16.088665 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-e0cc0347-8abf-451c-820a-00d4c346e9d9 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:16.089870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e0cc0347-8abf-451c-820a-00d4c346e9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:16.188385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 662/1333] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:18:16.224261 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7159bc-3623-4d06-9c37-ca6fd477e3b5 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:16.225776 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2f7159bc-3623-4d06-9c37-ca6fd477e3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:16.330257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7159bc-3623-4d06-9c37-ca6fd477e3b5 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:16.401779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6489a6-aa20-4ce5-b3fe-1cda2d00fbbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:16.402422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6489a6-aa20-4ce5-b3fe-1cda2d00fbbf service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:16.402703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6489a6-aa20-4ce5-b3fe-1cda2d00fbbf service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:16.402975 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6489a6-aa20-4ce5-b3fe-1cda2d00fbbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:16.404619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3f6489a6-aa20-4ce5-b3fe-1cda2d00fbbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:16.508487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6489a6-aa20-4ce5-b3fe-1cda2d00fbbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:16.509924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6489a6-aa20-4ce5-b3fe-1cda2d00fbbf service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:16.527119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 672/1334] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:18.849452 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31904931-9b43-4e0e-8630-1c77ce7f3404 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:18.849980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31904931-9b43-4e0e-8630-1c77ce7f3404 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:18.849980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31904931-9b43-4e0e-8630-1c77ce7f3404 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:18.851340 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 663/1335] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:18.856716 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c6cdf-6bd9-4e04-a55f-ae61f5685765 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:18.857111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c6cdf-6bd9-4e04-a55f-ae61f5685765 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:18.857275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c6cdf-6bd9-4e04-a55f-ae61f5685765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:18.922285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-687c6cdf-6bd9-4e04-a55f-ae61f5685765 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:18.923468 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-687c6cdf-6bd9-4e04-a55f-ae61f5685765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:19.015195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 673/1336] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:19.053274 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31904931-9b43-4e0e-8630-1c77ce7f3404 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:19.055528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-31904931-9b43-4e0e-8630-1c77ce7f3404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:19.153199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31904931-9b43-4e0e-8630-1c77ce7f3404 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:19.223191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bf4cf-1852-45ad-ae34-ca5c81000890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:19.223835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bf4cf-1852-45ad-ae34-ca5c81000890 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:19.224149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bf4cf-1852-45ad-ae34-ca5c81000890 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:19.224422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bf4cf-1852-45ad-ae34-ca5c81000890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:19.226683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-437bf4cf-1852-45ad-ae34-ca5c81000890 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:19.329610 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bf4cf-1852-45ad-ae34-ca5c81000890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:19.330930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bf4cf-1852-45ad-ae34-ca5c81000890 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:19.348478 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 664/1337] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:21.941316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2bf84-da7f-4eea-94da-44523305e4d8 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:21.941844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2bf84-da7f-4eea-94da-44523305e4d8 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:21.941967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2bf84-da7f-4eea-94da-44523305e4d8 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:21.943233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 674/1338] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:21.949281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d52e0-bc32-45ed-ac56-a81b83e4c5ce None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:21.949692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d52e0-bc32-45ed-ac56-a81b83e4c5ce None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:21.949835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d52e0-bc32-45ed-ac56-a81b83e4c5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:22.011650 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-189d52e0-bc32-45ed-ac56-a81b83e4c5ce None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:22.012790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-189d52e0-bc32-45ed-ac56-a81b83e4c5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:22.105107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 665/1339] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:22.139915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2bf84-da7f-4eea-94da-44523305e4d8 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:22.141419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-32f2bf84-da7f-4eea-94da-44523305e4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:22.239223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2bf84-da7f-4eea-94da-44523305e4d8 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:22.308323 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855ed6e-58ef-467f-ba2e-059436b331aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:22.308904 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855ed6e-58ef-467f-ba2e-059436b331aa service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:22.309189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855ed6e-58ef-467f-ba2e-059436b331aa service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:22.309468 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855ed6e-58ef-467f-ba2e-059436b331aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:22.311076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f855ed6e-58ef-467f-ba2e-059436b331aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:22.412745 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855ed6e-58ef-467f-ba2e-059436b331aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:22.414683 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855ed6e-58ef-467f-ba2e-059436b331aa service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:22.432278 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 675/1340] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:26.998672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528768e-66f6-4e38-94db-f269d4ce8cf9 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:26.999310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528768e-66f6-4e38-94db-f269d4ce8cf9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:26.999413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528768e-66f6-4e38-94db-f269d4ce8cf9 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:27.000841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 666/1341] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:27.005954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7d496-2581-4700-9ccf-6280ddeba114 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:27.006420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7d496-2581-4700-9ccf-6280ddeba114 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:27.006674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7d496-2581-4700-9ccf-6280ddeba114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:27.075019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e1a7d496-2581-4700-9ccf-6280ddeba114 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:27.076356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e1a7d496-2581-4700-9ccf-6280ddeba114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:27.176999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 676/1342] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:27.216246 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528768e-66f6-4e38-94db-f269d4ce8cf9 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:27.218362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-2528768e-66f6-4e38-94db-f269d4ce8cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:27.312301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528768e-66f6-4e38-94db-f269d4ce8cf9 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:27.315780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6ce52-8ff3-428d-9c5e-04b0b8e1733c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:27.316477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6ce52-8ff3-428d-9c5e-04b0b8e1733c service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:27.316787 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6ce52-8ff3-428d-9c5e-04b0b8e1733c service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:27.317065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6ce52-8ff3-428d-9c5e-04b0b8e1733c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:27.326725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f6ce52-8ff3-428d-9c5e-04b0b8e1733c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:27.328198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f6ce52-8ff3-428d-9c5e-04b0b8e1733c service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:27.338338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 667/1343] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:30.168445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88e238-f9a8-49d2-8e91-dec7208b6424 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:30.168954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88e238-f9a8-49d2-8e91-dec7208b6424 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:30.168954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88e238-f9a8-49d2-8e91-dec7208b6424 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:30.170268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 677/1344] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:30.175530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6b33a-7bc2-4929-b0ad-c171251dcb5c None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:30.175984 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6b33a-7bc2-4929-b0ad-c171251dcb5c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:30.176143 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6b33a-7bc2-4929-b0ad-c171251dcb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:30.202262 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-5ee6b33a-7bc2-4929-b0ad-c171251dcb5c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:30.203436 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5ee6b33a-7bc2-4929-b0ad-c171251dcb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:30.278020 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 668/1345] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:30.314569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88e238-f9a8-49d2-8e91-dec7208b6424 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:30.325718 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88e238-f9a8-49d2-8e91-dec7208b6424 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:30.329226 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ae6be-d822-4cbd-94eb-3e943d1b4bdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:30.329869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ae6be-d822-4cbd-94eb-3e943d1b4bdb service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:30.330152 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ae6be-d822-4cbd-94eb-3e943d1b4bdb service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:30.330429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ae6be-d822-4cbd-94eb-3e943d1b4bdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:30.339860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ae6be-d822-4cbd-94eb-3e943d1b4bdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:30.341317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ae6be-d822-4cbd-94eb-3e943d1b4bdb service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:30.351765 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 678/1346] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:34.583409 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eed71-3238-4b7a-834f-8720f5c98def None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:34.583829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eed71-3238-4b7a-834f-8720f5c98def None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:34.583829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eed71-3238-4b7a-834f-8720f5c98def None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:34.585274 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 669/1347] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:34.590858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d275de-915c-4ad0-8fa6-8c52a3491529 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:34.591295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d275de-915c-4ad0-8fa6-8c52a3491529 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:34.591418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d275de-915c-4ad0-8fa6-8c52a3491529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:34.609658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-67d275de-915c-4ad0-8fa6-8c52a3491529 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:34.611419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-67d275de-915c-4ad0-8fa6-8c52a3491529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:34.620016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 679/1348] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:34.657331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eed71-3238-4b7a-834f-8720f5c98def None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:34.667692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eed71-3238-4b7a-834f-8720f5c98def None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:34.671089 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437da515-2c2f-469b-bb3e-28233507f320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:34.671717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437da515-2c2f-469b-bb3e-28233507f320 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:34.672033 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437da515-2c2f-469b-bb3e-28233507f320 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:34.672309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437da515-2c2f-469b-bb3e-28233507f320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:34.681933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437da515-2c2f-469b-bb3e-28233507f320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:34.683344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437da515-2c2f-469b-bb3e-28233507f320 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:34.692462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 670/1349] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:40.051748 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e5619-bb8d-419e-9d75-96530011ca4a None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:40.053321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e5619-bb8d-419e-9d75-96530011ca4a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:40.054378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e5619-bb8d-419e-9d75-96530011ca4a None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:40.056196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 680/1350] 10.222.0.44 () {54 vars in 921 bytes} [Mon May 9 04:18:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:40.066551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aeb14f-b031-4f81-be32-b3f04a07f0a6 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:40.066996 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aeb14f-b031-4f81-be32-b3f04a07f0a6 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:40.067107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aeb14f-b031-4f81-be32-b3f04a07f0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:40.145948 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-a0aeb14f-b031-4f81-be32-b3f04a07f0a6 None None] MFA Rules not processed for user `d41e05b7ef954dd596c0134b65434dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:40.147574 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a0aeb14f-b031-4f81-be32-b3f04a07f0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:40.221606 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 671/1351] 10.222.0.44 () {58 vars in 1018 bytes} [Mon May 9 04:18:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:44.636075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81b94b-9268-479a-a10e-3c59c0637117 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:44.636666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81b94b-9268-479a-a10e-3c59c0637117 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:44.636666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81b94b-9268-479a-a10e-3c59c0637117 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:44.638248 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 681/1352] 10.222.0.44 () {54 vars in 921 bytes} [Mon May 9 04:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:44.643504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780abb05-8643-4a5a-8b03-d66b921c9cd7 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:44.643959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780abb05-8643-4a5a-8b03-d66b921c9cd7 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:44.644094 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780abb05-8643-4a5a-8b03-d66b921c9cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:44.662557 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-780abb05-8643-4a5a-8b03-d66b921c9cd7 None None] MFA Rules not processed for user `d41e05b7ef954dd596c0134b65434dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:44.664726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-780abb05-8643-4a5a-8b03-d66b921c9cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:44.673711 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 672/1353] 10.222.0.44 () {58 vars in 1018 bytes} [Mon May 9 04:18:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:46.215188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dc2cc-8b23-4bef-aa3b-27d76060b754 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:46.215705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dc2cc-8b23-4bef-aa3b-27d76060b754 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:46.215705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dc2cc-8b23-4bef-aa3b-27d76060b754 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:46.217469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 682/1354] 10.222.0.44 () {54 vars in 921 bytes} [Mon May 9 04:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:46.223424 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdba303-4b9c-4543-beba-853a659039fc None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:46.223869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdba303-4b9c-4543-beba-853a659039fc None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:46.224263 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdba303-4b9c-4543-beba-853a659039fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:46.241950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-ccdba303-4b9c-4543-beba-853a659039fc None None] MFA Rules not processed for user `d41e05b7ef954dd596c0134b65434dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:46.243123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-ccdba303-4b9c-4543-beba-853a659039fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:46.251490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 673/1355] 10.222.0.44 () {58 vars in 1018 bytes} [Mon May 9 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:18:50.128277 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deaf62b-3130-4807-be14-ea78f4153ae9 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:50.128889 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deaf62b-3130-4807-be14-ea78f4153ae9 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:50.128889 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deaf62b-3130-4807-be14-ea78f4153ae9 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:50.132111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 683/1356] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:50.136232 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f896b-d6d8-4ae4-a24e-b4b7bfcfc304 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:50.136637 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f896b-d6d8-4ae4-a24e-b4b7bfcfc304 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:50.136798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f896b-d6d8-4ae4-a24e-b4b7bfcfc304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:50.154070 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-e98f896b-d6d8-4ae4-a24e-b4b7bfcfc304 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:50.155362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e98f896b-d6d8-4ae4-a24e-b4b7bfcfc304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:50.163612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 674/1357] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:50.214459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7af25a-6066-4c2f-a46b-39ecafc6ee8a None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:50.215124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7af25a-6066-4c2f-a46b-39ecafc6ee8a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:50.215671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7af25a-6066-4c2f-a46b-39ecafc6ee8a None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:50.217369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 684/1358] 10.222.0.44 () {54 vars in 961 bytes} [Mon May 9 04:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:50.231644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e201b0d-9d5f-436e-a451-6b5374003bf5 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:50.231950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e201b0d-9d5f-436e-a451-6b5374003bf5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:50.232502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e201b0d-9d5f-436e-a451-6b5374003bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:50.281757 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-6e201b0d-9d5f-436e-a451-6b5374003bf5 None None] MFA Rules not processed for user `faf6107735424ff7901614953e7df403`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:50.282919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-6e201b0d-9d5f-436e-a451-6b5374003bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:50.364484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 675/1359] 10.222.0.44 () {58 vars in 1058 bytes} [Mon May 9 04:18:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:50.373884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7af25a-6066-4c2f-a46b-39ecafc6ee8a None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:50.389006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7af25a-6066-4c2f-a46b-39ecafc6ee8a None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:50.394306 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb55efe-530b-4ab6-8662-6425910c203a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:50.395123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb55efe-530b-4ab6-8662-6425910c203a service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:50.395414 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb55efe-530b-4ab6-8662-6425910c203a service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:50.395708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb55efe-530b-4ab6-8662-6425910c203a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:50.404398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb55efe-530b-4ab6-8662-6425910c203a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:50.406140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb55efe-530b-4ab6-8662-6425910c203a service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:50.417032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 685/1360] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:18:52.915259 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c205c-21ef-4d66-b7fa-bfe57b47774a None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:52.915802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c205c-21ef-4d66-b7fa-bfe57b47774a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:52.915802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c205c-21ef-4d66-b7fa-bfe57b47774a None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:52.917442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 676/1361] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:52.922552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65d25f-9769-40b3-8f47-53d9d3a43fc4 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:52.922953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65d25f-9769-40b3-8f47-53d9d3a43fc4 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:52.923110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65d25f-9769-40b3-8f47-53d9d3a43fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:52.943276 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-4e65d25f-9769-40b3-8f47-53d9d3a43fc4 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:52.945759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4e65d25f-9769-40b3-8f47-53d9d3a43fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:52.954199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 686/1362] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:52.997703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c205c-21ef-4d66-b7fa-bfe57b47774a None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:53.012336 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c205c-21ef-4d66-b7fa-bfe57b47774a None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:53.017877 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55d7a7-e388-486e-87a6-e36d915e62e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:53.018512 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55d7a7-e388-486e-87a6-e36d915e62e4 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:53.018797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55d7a7-e388-486e-87a6-e36d915e62e4 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:53.019076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55d7a7-e388-486e-87a6-e36d915e62e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:53.027955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55d7a7-e388-486e-87a6-e36d915e62e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:53.029551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55d7a7-e388-486e-87a6-e36d915e62e4 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:53.039761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 677/1363] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:55.550051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dbdb7-61b7-4fcb-8fd3-11bda9c0c8a3 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:55.550558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dbdb7-61b7-4fcb-8fd3-11bda9c0c8a3 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:55.550653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dbdb7-61b7-4fcb-8fd3-11bda9c0c8a3 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:55.551972 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 687/1364] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:55.557420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c0018-9d10-4577-b6ac-f42a5852f2fa None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:55.557854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c0018-9d10-4577-b6ac-f42a5852f2fa None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:55.558109 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c0018-9d10-4577-b6ac-f42a5852f2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:55.575500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d92c0018-9d10-4577-b6ac-f42a5852f2fa None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:55.576728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d92c0018-9d10-4577-b6ac-f42a5852f2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:55.585146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 678/1365] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:55.630099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d86440-07ad-4b77-a486-9b1b3c439a99 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:55.630390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d86440-07ad-4b77-a486-9b1b3c439a99 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:55.630669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d86440-07ad-4b77-a486-9b1b3c439a99 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:55.635447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 688/1366] 10.222.0.44 () {54 vars in 961 bytes} [Mon May 9 04:18:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:55.641210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3546b2-e6f9-4dbc-84ac-9f97f4fa31a6 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:55.641505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3546b2-e6f9-4dbc-84ac-9f97f4fa31a6 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:55.641786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3546b2-e6f9-4dbc-84ac-9f97f4fa31a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:55.660300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-0f3546b2-e6f9-4dbc-84ac-9f97f4fa31a6 None None] MFA Rules not processed for user `faf6107735424ff7901614953e7df403`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:55.661515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0f3546b2-e6f9-4dbc-84ac-9f97f4fa31a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:55.673815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 679/1367] 10.222.0.44 () {58 vars in 1058 bytes} [Mon May 9 04:18:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:18:55.681557 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d86440-07ad-4b77-a486-9b1b3c439a99 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:55.696846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d86440-07ad-4b77-a486-9b1b3c439a99 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:55.702252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e5019-3c73-4e93-b45e-d4ebfdec4260 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:55.702890 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e5019-3c73-4e93-b45e-d4ebfdec4260 service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:55.703172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e5019-3c73-4e93-b45e-d4ebfdec4260 service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:55.703447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e5019-3c73-4e93-b45e-d4ebfdec4260 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:55.712905 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e5019-3c73-4e93-b45e-d4ebfdec4260 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:55.714573 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e5019-3c73-4e93-b45e-d4ebfdec4260 service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:55.746146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 689/1368] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:58.085118 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5daa1-272b-4294-8cd5-3250a48783d1 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:58.086086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5daa1-272b-4294-8cd5-3250a48783d1 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:58.086545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5daa1-272b-4294-8cd5-3250a48783d1 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:58.088141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 680/1369] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:58.093580 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dd2cd-235a-4a42-880b-b1e6a409c65e None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:58.094060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dd2cd-235a-4a42-880b-b1e6a409c65e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:58.094174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dd2cd-235a-4a42-880b-b1e6a409c65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:58.112001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-a69dd2cd-235a-4a42-880b-b1e6a409c65e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:58.113194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a69dd2cd-235a-4a42-880b-b1e6a409c65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:58.121804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 690/1370] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:58.167252 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5daa1-272b-4294-8cd5-3250a48783d1 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:58.182519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5daa1-272b-4294-8cd5-3250a48783d1 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:58.188073 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a4e9d-b11e-4991-be15-94beeec40bb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:58.188729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a4e9d-b11e-4991-be15-94beeec40bb4 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:58.189015 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a4e9d-b11e-4991-be15-94beeec40bb4 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:58.189292 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a4e9d-b11e-4991-be15-94beeec40bb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:58.198863 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a4e9d-b11e-4991-be15-94beeec40bb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:58.200618 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a4e9d-b11e-4991-be15-94beeec40bb4 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:58.211134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 681/1371] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:18:58.488081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d3ce2-3a11-43fd-8855-6e9c3797f14b None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:58.488366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d3ce2-3a11-43fd-8855-6e9c3797f14b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:58.488828 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d3ce2-3a11-43fd-8855-6e9c3797f14b None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:58.490418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 691/1372] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:18:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:58.496431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d88665-3c1a-49f3-97d4-9b91b4ad79e3 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:58.496720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d88665-3c1a-49f3-97d4-9b91b4ad79e3 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:58.497166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d88665-3c1a-49f3-97d4-9b91b4ad79e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:58.516233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-10d88665-3c1a-49f3-97d4-9b91b4ad79e3 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:58.517694 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-10d88665-3c1a-49f3-97d4-9b91b4ad79e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:58.526880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 682/1373] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:58.870309 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93367a15-d327-4c33-8fef-d890f9a3ecaa None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:58.871390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93367a15-d327-4c33-8fef-d890f9a3ecaa None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:58.871931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93367a15-d327-4c33-8fef-d890f9a3ecaa None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:58.874429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 692/1374] 10.222.0.44 () {54 vars in 963 bytes} [Mon May 9 04:18:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:18:58.880323 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa1bbb-634a-42c7-a878-b9e871c0074d None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:58.880764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa1bbb-634a-42c7-a878-b9e871c0074d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:58.881139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa1bbb-634a-42c7-a878-b9e871c0074d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:58.934598 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-b0aa1bbb-634a-42c7-a878-b9e871c0074d None None] MFA Rules not processed for user `72caf90f411f495bb22aa85df4d25eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:18:58.936027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b0aa1bbb-634a-42c7-a878-b9e871c0074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:18:58.995585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 683/1375] 10.222.0.44 () {58 vars in 1060 bytes} [Mon May 9 04:18:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:18:59.003044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93367a15-d327-4c33-8fef-d890f9a3ecaa None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:18:59.013463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93367a15-d327-4c33-8fef-d890f9a3ecaa None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:18:59.016948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2ed32-dd18-4ba9-8de9-07349a8dbd43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:18:59.017687 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2ed32-dd18-4ba9-8de9-07349a8dbd43 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:18:59.018062 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2ed32-dd18-4ba9-8de9-07349a8dbd43 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:18:59.018432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2ed32-dd18-4ba9-8de9-07349a8dbd43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:18:59.027666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2ed32-dd18-4ba9-8de9-07349a8dbd43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:18:59.029245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2ed32-dd18-4ba9-8de9-07349a8dbd43 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:18:59.039515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 693/1376] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:18:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:00.487850 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6e50e-9be0-4696-9994-b6ab5a1840f2 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:00.488427 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6e50e-9be0-4696-9994-b6ab5a1840f2 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:00.488427 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6e50e-9be0-4696-9994-b6ab5a1840f2 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:00.490187 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 684/1377] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:00.496767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b64103-319e-430b-9b1e-87058d9ea3a8 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:00.497231 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b64103-319e-430b-9b1e-87058d9ea3a8 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:00.497457 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b64103-319e-430b-9b1e-87058d9ea3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:00.515796 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-05b64103-319e-430b-9b1e-87058d9ea3a8 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:00.517068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-05b64103-319e-430b-9b1e-87058d9ea3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:00.525246 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 694/1378] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:00.570996 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6e50e-9be0-4696-9994-b6ab5a1840f2 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:00.585959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6e50e-9be0-4696-9994-b6ab5a1840f2 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:00.591563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1905f9-75c8-4a2a-b391-f980ae4f022f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:00.592261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1905f9-75c8-4a2a-b391-f980ae4f022f service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:00.592547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1905f9-75c8-4a2a-b391-f980ae4f022f service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:00.592833 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1905f9-75c8-4a2a-b391-f980ae4f022f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:00.602443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1905f9-75c8-4a2a-b391-f980ae4f022f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:00.604144 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1905f9-75c8-4a2a-b391-f980ae4f022f service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:00.614459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 685/1379] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:02.902441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d70ce-0f70-4d67-aa24-53c5d928542f None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:02.902990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d70ce-0f70-4d67-aa24-53c5d928542f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:02.903275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d70ce-0f70-4d67-aa24-53c5d928542f None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:02.904760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 695/1380] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:02.911083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec33d34-0b2c-4efb-8a08-eb8242085c6e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:02.911546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec33d34-0b2c-4efb-8a08-eb8242085c6e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:02.911630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec33d34-0b2c-4efb-8a08-eb8242085c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:02.932813 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-cec33d34-0b2c-4efb-8a08-eb8242085c6e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:02.933984 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-cec33d34-0b2c-4efb-8a08-eb8242085c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:02.942892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 686/1381] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:02.987200 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d70ce-0f70-4d67-aa24-53c5d928542f None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:03.002093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d70ce-0f70-4d67-aa24-53c5d928542f None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:03.008442 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341a939-f0c1-4ac0-9add-4c6e4a71c51f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:03.009240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341a939-f0c1-4ac0-9add-4c6e4a71c51f service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:03.009566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341a939-f0c1-4ac0-9add-4c6e4a71c51f service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:03.009934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341a939-f0c1-4ac0-9add-4c6e4a71c51f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:03.018319 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341a939-f0c1-4ac0-9add-4c6e4a71c51f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:03.020033 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341a939-f0c1-4ac0-9add-4c6e4a71c51f service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:03.029831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 696/1382] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:05.330973 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4bcff-75ea-4c29-9e0c-ec3a5614d3e8 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:05.331728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4bcff-75ea-4c29-9e0c-ec3a5614d3e8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:05.331728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4bcff-75ea-4c29-9e0c-ec3a5614d3e8 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:05.333302 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 687/1383] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:05.339204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e9a31-10f5-45cd-88cb-0736c11ae615 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:05.339627 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e9a31-10f5-45cd-88cb-0736c11ae615 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:05.339764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e9a31-10f5-45cd-88cb-0736c11ae615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:05.357966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-626e9a31-10f5-45cd-88cb-0736c11ae615 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:05.359370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-626e9a31-10f5-45cd-88cb-0736c11ae615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:05.367383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 697/1384] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:05.409792 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4bcff-75ea-4c29-9e0c-ec3a5614d3e8 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:05.424829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4bcff-75ea-4c29-9e0c-ec3a5614d3e8 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:05.430895 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d405947-497a-4b50-900d-d80b973616b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:05.431791 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d405947-497a-4b50-900d-d80b973616b1 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:05.432314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d405947-497a-4b50-900d-d80b973616b1 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:05.432764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d405947-497a-4b50-900d-d80b973616b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:05.441955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d405947-497a-4b50-900d-d80b973616b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:05.444091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d405947-497a-4b50-900d-d80b973616b1 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:05.455703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 688/1385] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:07.726520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb797e-5c0c-4705-909e-33ddc9597de7 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:07.726520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb797e-5c0c-4705-909e-33ddc9597de7 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:07.726520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb797e-5c0c-4705-909e-33ddc9597de7 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:07.727586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 698/1386] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:07.733110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ec6a8-fa9d-4b9a-ae4c-e1f78c1c315b None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:07.733735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ec6a8-fa9d-4b9a-ae4c-e1f78c1c315b None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:07.733735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ec6a8-fa9d-4b9a-ae4c-e1f78c1c315b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:07.752602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-0f0ec6a8-fa9d-4b9a-ae4c-e1f78c1c315b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:07.753807 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0f0ec6a8-fa9d-4b9a-ae4c-e1f78c1c315b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:07.762278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 689/1387] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:07.808696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb797e-5c0c-4705-909e-33ddc9597de7 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:07.826206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb797e-5c0c-4705-909e-33ddc9597de7 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:07.833499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651b4f4-88a3-4e09-b900-0b6a1c57f17c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:07.834429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651b4f4-88a3-4e09-b900-0b6a1c57f17c service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:07.834796 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651b4f4-88a3-4e09-b900-0b6a1c57f17c service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:07.835160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651b4f4-88a3-4e09-b900-0b6a1c57f17c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:07.845322 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651b4f4-88a3-4e09-b900-0b6a1c57f17c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:07.847335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651b4f4-88a3-4e09-b900-0b6a1c57f17c service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:07.859782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 699/1388] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:19:10.110591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4e2f3-5fdb-43b9-a37b-f4dfca0cbb88 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:10.111300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4e2f3-5fdb-43b9-a37b-f4dfca0cbb88 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:10.111300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4e2f3-5fdb-43b9-a37b-f4dfca0cbb88 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:10.112419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 690/1389] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:10.118713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40dbd5e-6eb5-4708-bf2b-929e1b82c196 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:10.119184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40dbd5e-6eb5-4708-bf2b-929e1b82c196 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:10.119463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40dbd5e-6eb5-4708-bf2b-929e1b82c196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:10.141633 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-b40dbd5e-6eb5-4708-bf2b-929e1b82c196 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:10.143578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b40dbd5e-6eb5-4708-bf2b-929e1b82c196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:10.154361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 700/1390] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:10.199560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4e2f3-5fdb-43b9-a37b-f4dfca0cbb88 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:10.214023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4e2f3-5fdb-43b9-a37b-f4dfca0cbb88 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:10.221832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addeece-e364-4035-bb9b-6f6d1bc4525a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:10.222481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addeece-e364-4035-bb9b-6f6d1bc4525a service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:10.222757 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addeece-e364-4035-bb9b-6f6d1bc4525a service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:10.223026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addeece-e364-4035-bb9b-6f6d1bc4525a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:10.234222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2addeece-e364-4035-bb9b-6f6d1bc4525a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:10.235846 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2addeece-e364-4035-bb9b-6f6d1bc4525a service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:10.246575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 691/1391] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:19:12.932855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5039b65-0355-4235-b5ea-f322479898f0 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:12.933458 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5039b65-0355-4235-b5ea-f322479898f0 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:12.933458 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5039b65-0355-4235-b5ea-f322479898f0 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:12.934812 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 701/1392] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:12.940087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f5c31-1a85-47d4-bfc3-b8e61c3a3c95 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:12.940558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f5c31-1a85-47d4-bfc3-b8e61c3a3c95 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:12.940669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f5c31-1a85-47d4-bfc3-b8e61c3a3c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:12.959806 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-dd5f5c31-1a85-47d4-bfc3-b8e61c3a3c95 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:12.961349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-dd5f5c31-1a85-47d4-bfc3-b8e61c3a3c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:12.976282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 692/1393] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:13.016660 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5039b65-0355-4235-b5ea-f322479898f0 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:13.031877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5039b65-0355-4235-b5ea-f322479898f0 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:13.038260 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6ec64-4cae-4261-ad4b-5781dcbe20f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:13.038911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6ec64-4cae-4261-ad4b-5781dcbe20f4 service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:13.039190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6ec64-4cae-4261-ad4b-5781dcbe20f4 service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:13.039461 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6ec64-4cae-4261-ad4b-5781dcbe20f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:13.048066 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6ec64-4cae-4261-ad4b-5781dcbe20f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:13.049661 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6ec64-4cae-4261-ad4b-5781dcbe20f4 service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:13.059538 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 702/1394] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:14.761202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23451fac-040f-4648-a6d4-6a999ecb9145 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:14.761620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23451fac-040f-4648-a6d4-6a999ecb9145 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:14.761892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23451fac-040f-4648-a6d4-6a999ecb9145 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:14.763328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 693/1395] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:14.768683 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea81667-53e0-4a85-ab35-7c5acb6ae510 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:14.769087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea81667-53e0-4a85-ab35-7c5acb6ae510 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:14.769244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea81667-53e0-4a85-ab35-7c5acb6ae510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:14.787020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-aea81667-53e0-4a85-ab35-7c5acb6ae510 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:14.788838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-aea81667-53e0-4a85-ab35-7c5acb6ae510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:14.796488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 703/1396] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:14.851907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dd63a-d81d-47af-b324-a51104769ccc None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:14.852199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dd63a-d81d-47af-b324-a51104769ccc None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:14.852474 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dd63a-d81d-47af-b324-a51104769ccc None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:14.853843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 694/1397] 10.222.0.44 () {54 vars in 962 bytes} [Mon May 9 04:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:14.859100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea9d1b-7d33-4849-8061-fd043b77e265 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:14.859392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea9d1b-7d33-4849-8061-fd043b77e265 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:14.859675 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea9d1b-7d33-4849-8061-fd043b77e265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:14.907028 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2cea9d1b-7d33-4849-8061-fd043b77e265 None None] MFA Rules not processed for user `295bb46627e043989e6dba2805607c99`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:14.908251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2cea9d1b-7d33-4849-8061-fd043b77e265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:14.965922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 704/1398] 10.222.0.44 () {58 vars in 1059 bytes} [Mon May 9 04:19:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:14.973351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65dd63a-d81d-47af-b324-a51104769ccc None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:14.984405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65dd63a-d81d-47af-b324-a51104769ccc None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:14.987873 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e77c1-8f8c-431e-88ef-eadd5c1243b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '295bb46627e043989e6dba2805607c99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:14.988512 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e77c1-8f8c-431e-88ef-eadd5c1243b2 service glance] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:14.988797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e77c1-8f8c-431e-88ef-eadd5c1243b2 service glance] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:14.989063 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e77c1-8f8c-431e-88ef-eadd5c1243b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:14.997965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e77c1-8f8c-431e-88ef-eadd5c1243b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:14.999635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e77c1-8f8c-431e-88ef-eadd5c1243b2 service glance] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:15.009076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 695/1399] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:15.306530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5c6a8-4463-4364-bd8e-963d1f676fd2 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:15.306816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5c6a8-4463-4364-bd8e-963d1f676fd2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:15.307091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5c6a8-4463-4364-bd8e-963d1f676fd2 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:15.308483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 705/1400] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:15.313876 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efccac-e309-4b80-a899-bcc5dc72b04a None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:15.314169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efccac-e309-4b80-a899-bcc5dc72b04a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:15.314448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efccac-e309-4b80-a899-bcc5dc72b04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:15.332738 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-26efccac-e309-4b80-a899-bcc5dc72b04a None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:15.333922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-26efccac-e309-4b80-a899-bcc5dc72b04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:15.341843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 696/1401] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:15.386923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5c6a8-4463-4364-bd8e-963d1f676fd2 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:15.401650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5c6a8-4463-4364-bd8e-963d1f676fd2 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:15.410773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f6828-a10a-41c7-9b44-b1df51b60369 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:15.411423 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f6828-a10a-41c7-9b44-b1df51b60369 service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:15.411699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f6828-a10a-41c7-9b44-b1df51b60369 service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:15.412000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f6828-a10a-41c7-9b44-b1df51b60369 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:15.420655 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f6828-a10a-41c7-9b44-b1df51b60369 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:15.422316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f6828-a10a-41c7-9b44-b1df51b60369 service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:15.423524 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8a1c1-3364-4aba-a632-f37dc11117c2 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:15.423809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8a1c1-3364-4aba-a632-f37dc11117c2 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:15.424115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8a1c1-3364-4aba-a632-f37dc11117c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:15.431693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 706/1402] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:15.474054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-82a8a1c1-3364-4aba-a632-f37dc11117c2 None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:15.475476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-82a8a1c1-3364-4aba-a632-f37dc11117c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:15.544584 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 697/1403] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:19:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:19:15.591948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760d256-63f6-4077-ba5e-13b733ab4670 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:15.592239 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760d256-63f6-4077-ba5e-13b733ab4670 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:15.592519 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760d256-63f6-4077-ba5e-13b733ab4670 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:15.593938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 707/1404] 10.222.0.44 () {54 vars in 942 bytes} [Mon May 9 04:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:15.601470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8349cc1-4eb6-45fe-8ac1-fba766f11cc8 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:15.601759 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8349cc1-4eb6-45fe-8ac1-fba766f11cc8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:15.602038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8349cc1-4eb6-45fe-8ac1-fba766f11cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:15.656379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-b8349cc1-4eb6-45fe-8ac1-fba766f11cc8 None None] MFA Rules not processed for user `c46e7cc6407f44f08a659288e3ad0767`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:15.657597 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b8349cc1-4eb6-45fe-8ac1-fba766f11cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:15.727668 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 698/1405] 10.222.0.44 () {58 vars in 1039 bytes} [Mon May 9 04:19:15 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:15.736927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760d256-63f6-4077-ba5e-13b733ab4670 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:15.753730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760d256-63f6-4077-ba5e-13b733ab4670 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:15.762072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb29023-bb09-46fe-bcff-4e7869a549d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:15.762740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb29023-bb09-46fe-bcff-4e7869a549d6 service swift] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:15.763020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb29023-bb09-46fe-bcff-4e7869a549d6 service swift] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:15.763337 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb29023-bb09-46fe-bcff-4e7869a549d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:15.772668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb29023-bb09-46fe-bcff-4e7869a549d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:15.774310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb29023-bb09-46fe-bcff-4e7869a549d6 service swift] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:15.782703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 708/1406] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:19:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:19:17.699725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bcea5-0734-458a-aa82-9f7879565ce5 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:17.700334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bcea5-0734-458a-aa82-9f7879565ce5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:17.700334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bcea5-0734-458a-aa82-9f7879565ce5 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:17.701671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 699/1407] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:17.713941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190eadda-13d6-4b0c-a678-fe8ff1429314 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:17.715408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190eadda-13d6-4b0c-a678-fe8ff1429314 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:17.716492 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190eadda-13d6-4b0c-a678-fe8ff1429314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:17.741652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-190eadda-13d6-4b0c-a678-fe8ff1429314 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:17.743105 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-190eadda-13d6-4b0c-a678-fe8ff1429314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:17.750962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 709/1408] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:17.796777 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bcea5-0734-458a-aa82-9f7879565ce5 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:17.811368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bcea5-0734-458a-aa82-9f7879565ce5 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:17.817346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a417f-2bc3-4dbb-b02e-5e8822a3e6f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:17.819064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a417f-2bc3-4dbb-b02e-5e8822a3e6f3 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:17.819371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a417f-2bc3-4dbb-b02e-5e8822a3e6f3 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:17.819718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a417f-2bc3-4dbb-b02e-5e8822a3e6f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:17.830556 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a417f-2bc3-4dbb-b02e-5e8822a3e6f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:17.832560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a417f-2bc3-4dbb-b02e-5e8822a3e6f3 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:17.842818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 700/1409] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:39.287865 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c8e97-9ecf-4909-aa81-ec2612f6751f None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:39.288537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c8e97-9ecf-4909-aa81-ec2612f6751f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:39.288537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c8e97-9ecf-4909-aa81-ec2612f6751f None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:39.289792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 710/1410] 10.222.0.44 () {54 vars in 918 bytes} [Mon May 9 04:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:39.295056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d6ede-a121-4307-8544-4ea4ee91d993 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:39.295554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d6ede-a121-4307-8544-4ea4ee91d993 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:39.295666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d6ede-a121-4307-8544-4ea4ee91d993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:39.313474 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-967d6ede-a121-4307-8544-4ea4ee91d993 None None] MFA Rules not processed for user `d41e05b7ef954dd596c0134b65434dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:39.314646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-967d6ede-a121-4307-8544-4ea4ee91d993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:39.324583 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 701/1411] 10.222.0.44 () {58 vars in 1015 bytes} [Mon May 9 04:19:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:39.338893 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0851c-f324-4416-9a49-5e4b58671a33 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:39.339182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0851c-f324-4416-9a49-5e4b58671a33 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:39.339480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0851c-f324-4416-9a49-5e4b58671a33 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:39.340861 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 711/1412] 10.222.0.44 () {54 vars in 918 bytes} [Mon May 9 04:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:39.345413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944119c-3e21-4208-b253-220bff847f77 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:39.345699 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944119c-3e21-4208-b253-220bff847f77 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:39.345976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944119c-3e21-4208-b253-220bff847f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:39.364342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-1944119c-3e21-4208-b253-220bff847f77 None None] MFA Rules not processed for user `faf6107735424ff7901614953e7df403`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:39.366406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1944119c-3e21-4208-b253-220bff847f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:39.375124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 702/1413] 10.222.0.44 () {58 vars in 1015 bytes} [Mon May 9 04:19:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:42.054906 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4cac8-585b-4c15-bcfa-8c07e045fa80 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:42.055521 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4cac8-585b-4c15-bcfa-8c07e045fa80 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:42.055521 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4cac8-585b-4c15-bcfa-8c07e045fa80 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:42.056764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:42.062163 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd504bd8-ae23-489c-9f4f-94c7f4716b99 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:42.062594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd504bd8-ae23-489c-9f4f-94c7f4716b99 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:42.062700 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd504bd8-ae23-489c-9f4f-94c7f4716b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:42.080231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-bd504bd8-ae23-489c-9f4f-94c7f4716b99 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:42.081400 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-bd504bd8-ae23-489c-9f4f-94c7f4716b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:42.089343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:19:42.131570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4cac8-585b-4c15-bcfa-8c07e045fa80 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:42.145271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4cac8-585b-4c15-bcfa-8c07e045fa80 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:42.150755 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec0126-577c-4d58-be5a-d62c916474d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:42.151383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec0126-577c-4d58-be5a-d62c916474d0 service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:42.151661 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec0126-577c-4d58-be5a-d62c916474d0 service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:42.151958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec0126-577c-4d58-be5a-d62c916474d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:42.159684 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec0126-577c-4d58-be5a-d62c916474d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:42.161300 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec0126-577c-4d58-be5a-d62c916474d0 service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:42.170400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 713/1416] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:19:43.152832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb195b-30f3-4f7c-a1f9-447d2146248e None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:43.153375 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb195b-30f3-4f7c-a1f9-447d2146248e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:43.153375 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb195b-30f3-4f7c-a1f9-447d2146248e None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:43.154661 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 04:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:43.159746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b1327-6a23-4e73-b22e-3820868c5a8c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:43.160210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b1327-6a23-4e73-b22e-3820868c5a8c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:43.160322 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b1327-6a23-4e73-b22e-3820868c5a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:43.178478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-ee6b1327-6a23-4e73-b22e-3820868c5a8c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:43.179658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ee6b1327-6a23-4e73-b22e-3820868c5a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:43.187209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 04:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:43.353113 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb195b-30f3-4f7c-a1f9-447d2146248e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:43.369894 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb195b-30f3-4f7c-a1f9-447d2146248e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:43.377749 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6c50f-9464-4933-ba3b-4ce13ad8306c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:43.378392 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6c50f-9464-4933-ba3b-4ce13ad8306c service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:43.378676 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6c50f-9464-4933-ba3b-4ce13ad8306c service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:43.378949 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6c50f-9464-4933-ba3b-4ce13ad8306c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:43.388611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6c50f-9464-4933-ba3b-4ce13ad8306c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:43.390176 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6c50f-9464-4933-ba3b-4ce13ad8306c service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:43.399446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 705/1419] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:44.354123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc84ca-12bb-4b86-8d25-2d2760c7dc36 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:44.354691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc84ca-12bb-4b86-8d25-2d2760c7dc36 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:44.354691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc84ca-12bb-4b86-8d25-2d2760c7dc36 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:44.356081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 04:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:44.361024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b684880-e489-4249-b6a5-0e0b073c452e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:44.361443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b684880-e489-4249-b6a5-0e0b073c452e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:44.361584 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b684880-e489-4249-b6a5-0e0b073c452e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:44.379273 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-7b684880-e489-4249-b6a5-0e0b073c452e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:44.381101 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-7b684880-e489-4249-b6a5-0e0b073c452e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:44.388677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 04:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:44.538046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc84ca-12bb-4b86-8d25-2d2760c7dc36 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:44.551756 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc84ca-12bb-4b86-8d25-2d2760c7dc36 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:44.557659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e265d-9459-4357-adf0-c8991034ff2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:44.558288 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e265d-9459-4357-adf0-c8991034ff2a service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:44.558569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e265d-9459-4357-adf0-c8991034ff2a service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:44.558842 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e265d-9459-4357-adf0-c8991034ff2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:44.566586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e265d-9459-4357-adf0-c8991034ff2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:44.568762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e265d-9459-4357-adf0-c8991034ff2a service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:44.577824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 716/1422] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:19:45.500540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d515c03-919e-4ac4-8392-3bea8017901e None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:45.501141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d515c03-919e-4ac4-8392-3bea8017901e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:45.501141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d515c03-919e-4ac4-8392-3bea8017901e None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:45.502719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 04:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:45.508357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f74a94-4eee-4d96-8036-0a8f64eb4580 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:45.509010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f74a94-4eee-4d96-8036-0a8f64eb4580 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:45.509010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f74a94-4eee-4d96-8036-0a8f64eb4580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:45.527606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-a7f74a94-4eee-4d96-8036-0a8f64eb4580 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:45.529002 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a7f74a94-4eee-4d96-8036-0a8f64eb4580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:45.538475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 04:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:45.691506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d515c03-919e-4ac4-8392-3bea8017901e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:45.705593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d515c03-919e-4ac4-8392-3bea8017901e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:45.711621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e47d3-9bab-411e-af7d-acb5da20e273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:45.712292 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e47d3-9bab-411e-af7d-acb5da20e273 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:45.712572 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e47d3-9bab-411e-af7d-acb5da20e273 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:45.712845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e47d3-9bab-411e-af7d-acb5da20e273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:45.720631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e47d3-9bab-411e-af7d-acb5da20e273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:45.722174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e47d3-9bab-411e-af7d-acb5da20e273 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:45.731827 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 708/1425] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:46.656445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5b766-7a7c-45a9-9a02-7c049881d0d2 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:46.656991 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5b766-7a7c-45a9-9a02-7c049881d0d2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:46.656991 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5b766-7a7c-45a9-9a02-7c049881d0d2 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:46.658374 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 04:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:46.663241 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf5f03-b383-487c-abdf-cfa1dc975e76 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:46.663688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf5f03-b383-487c-abdf-cfa1dc975e76 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:46.663795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf5f03-b383-487c-abdf-cfa1dc975e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:46.681564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-51cf5f03-b383-487c-abdf-cfa1dc975e76 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:46.682726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-51cf5f03-b383-487c-abdf-cfa1dc975e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:46.690354 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 04:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:46.841469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5b766-7a7c-45a9-9a02-7c049881d0d2 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:46.855918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5b766-7a7c-45a9-9a02-7c049881d0d2 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:46.861290 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ccdc1-7ea2-4fc1-97b8-23b1fa919fd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:46.861924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ccdc1-7ea2-4fc1-97b8-23b1fa919fd0 service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:46.862210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ccdc1-7ea2-4fc1-97b8-23b1fa919fd0 service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:46.862487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ccdc1-7ea2-4fc1-97b8-23b1fa919fd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:46.870925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ccdc1-7ea2-4fc1-97b8-23b1fa919fd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:46.872539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ccdc1-7ea2-4fc1-97b8-23b1fa919fd0 service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:46.881892 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 719/1428] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:47.764294 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296441e-4ec3-4183-be2c-ebc84f7ef82f None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:47.764752 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296441e-4ec3-4183-be2c-ebc84f7ef82f None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:47.764867 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296441e-4ec3-4183-be2c-ebc84f7ef82f None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:47.766212 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 04:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:47.771104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f91fd-b3aa-42d0-a1e1-62aea0430a2e None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:47.771546 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f91fd-b3aa-42d0-a1e1-62aea0430a2e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:47.771674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f91fd-b3aa-42d0-a1e1-62aea0430a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:47.790788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-b43f91fd-b3aa-42d0-a1e1-62aea0430a2e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:47.792038 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b43f91fd-b3aa-42d0-a1e1-62aea0430a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:47.799680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 04:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:47.950099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296441e-4ec3-4183-be2c-ebc84f7ef82f None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:47.964831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296441e-4ec3-4183-be2c-ebc84f7ef82f None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:47.970048 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6594a581-1e8e-4472-bc89-b11d2a360750 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:47.970681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594a581-1e8e-4472-bc89-b11d2a360750 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:47.970965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594a581-1e8e-4472-bc89-b11d2a360750 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:47.971242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594a581-1e8e-4472-bc89-b11d2a360750 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:47.979135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594a581-1e8e-4472-bc89-b11d2a360750 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:47.980717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594a581-1e8e-4472-bc89-b11d2a360750 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:47.989688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 711/1431] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:48.855465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ecb13-9e31-4c8b-98ac-7e85e70c77e5 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:48.856021 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ecb13-9e31-4c8b-98ac-7e85e70c77e5 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:48.856021 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ecb13-9e31-4c8b-98ac-7e85e70c77e5 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:48.857422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 04:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:48.862487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d376a3b-2ab6-4152-a700-7695847bc3c1 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:48.862894 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d376a3b-2ab6-4152-a700-7695847bc3c1 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:48.863049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d376a3b-2ab6-4152-a700-7695847bc3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:48.882170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-4d376a3b-2ab6-4152-a700-7695847bc3c1 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:48.883338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-4d376a3b-2ab6-4152-a700-7695847bc3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:48.891035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 04:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:49.040751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ecb13-9e31-4c8b-98ac-7e85e70c77e5 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:49.054846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ecb13-9e31-4c8b-98ac-7e85e70c77e5 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:49.060088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be675ac-bf79-472d-a51f-2926de82ebcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:49.060713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be675ac-bf79-472d-a51f-2926de82ebcb service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:49.060989 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be675ac-bf79-472d-a51f-2926de82ebcb service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:49.061264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be675ac-bf79-472d-a51f-2926de82ebcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:49.069481 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be675ac-bf79-472d-a51f-2926de82ebcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:49.071025 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be675ac-bf79-472d-a51f-2926de82ebcb service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:49.079916 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 722/1434] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:49.951943 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7678e-8ce1-4f25-ab93-04bdca7d0b03 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:49.952510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7678e-8ce1-4f25-ab93-04bdca7d0b03 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:49.952510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7678e-8ce1-4f25-ab93-04bdca7d0b03 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:49.953862 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 04:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:49.959530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e2019-074c-489d-9f94-c7d16a29ea58 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:49.959975 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e2019-074c-489d-9f94-c7d16a29ea58 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:49.960157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e2019-074c-489d-9f94-c7d16a29ea58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:49.978603 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c19e2019-074c-489d-9f94-c7d16a29ea58 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:49.979789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c19e2019-074c-489d-9f94-c7d16a29ea58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:49.987532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 04:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:50.137870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7678e-8ce1-4f25-ab93-04bdca7d0b03 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:50.151643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7678e-8ce1-4f25-ab93-04bdca7d0b03 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:50.157107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98250a-dae4-4c26-9662-58741c67ef68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:50.157759 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98250a-dae4-4c26-9662-58741c67ef68 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:50.158037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98250a-dae4-4c26-9662-58741c67ef68 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:50.158309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98250a-dae4-4c26-9662-58741c67ef68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:50.166658 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98250a-dae4-4c26-9662-58741c67ef68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:50.168241 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98250a-dae4-4c26-9662-58741c67ef68 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:50.177141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 714/1437] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:51.046271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16c7fc-3f20-45ae-a60f-939878a81d89 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:51.046806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16c7fc-3f20-45ae-a60f-939878a81d89 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:51.046806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16c7fc-3f20-45ae-a60f-939878a81d89 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:51.048127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 04:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:51.053804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96ca7c-cf4f-4f3a-9270-b10a14e47f6e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:51.054363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96ca7c-cf4f-4f3a-9270-b10a14e47f6e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:51.054363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96ca7c-cf4f-4f3a-9270-b10a14e47f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:51.072425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-1b96ca7c-cf4f-4f3a-9270-b10a14e47f6e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:51.073617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1b96ca7c-cf4f-4f3a-9270-b10a14e47f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:51.084315 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 04:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:51.253144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16c7fc-3f20-45ae-a60f-939878a81d89 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:51.267822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16c7fc-3f20-45ae-a60f-939878a81d89 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:51.273412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9e1ce-c8d1-43cf-9baa-f1d3add2e6cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:51.274048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9e1ce-c8d1-43cf-9baa-f1d3add2e6cd service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:51.274334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9e1ce-c8d1-43cf-9baa-f1d3add2e6cd service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:51.274610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9e1ce-c8d1-43cf-9baa-f1d3add2e6cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:51.282488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9e1ce-c8d1-43cf-9baa-f1d3add2e6cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:51.284082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9e1ce-c8d1-43cf-9baa-f1d3add2e6cd service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:51.293285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 725/1440] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:53.493319 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bde67-a232-49ba-b7a8-73f72f31e009 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:53.493891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bde67-a232-49ba-b7a8-73f72f31e009 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:53.493891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bde67-a232-49ba-b7a8-73f72f31e009 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:53.495244 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:53.500623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf1af7-2296-4099-aeef-2b301f9a08d3 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:53.501073 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf1af7-2296-4099-aeef-2b301f9a08d3 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:53.501170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf1af7-2296-4099-aeef-2b301f9a08d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:53.519067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-32bf1af7-2296-4099-aeef-2b301f9a08d3 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:53.520323 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-32bf1af7-2296-4099-aeef-2b301f9a08d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:53.528095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:19:53.566145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5bde67-a232-49ba-b7a8-73f72f31e009 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:53.577184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5bde67-a232-49ba-b7a8-73f72f31e009 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:53.580619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e831b-d853-4a2d-b039-e540ac109814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:53.581247 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e831b-d853-4a2d-b039-e540ac109814 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:53.581526 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e831b-d853-4a2d-b039-e540ac109814 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:53.581800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e831b-d853-4a2d-b039-e540ac109814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:53.590531 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e831b-d853-4a2d-b039-e540ac109814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:53.591958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e831b-d853-4a2d-b039-e540ac109814 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:53.601516 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 717/1443] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:55.967119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff2d83-acf8-446b-8909-b128bc89c40c None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:55.967641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff2d83-acf8-446b-8909-b128bc89c40c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:55.967754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff2d83-acf8-446b-8909-b128bc89c40c None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:55.969070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:55.974450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600d52-fb1d-4549-8b7e-615e9be68721 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:55.974879 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600d52-fb1d-4549-8b7e-615e9be68721 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:55.975084 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600d52-fb1d-4549-8b7e-615e9be68721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:55.992459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-9a600d52-fb1d-4549-8b7e-615e9be68721 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:55.993631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9a600d52-fb1d-4549-8b7e-615e9be68721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:56.001147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:19:56.038537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff2d83-acf8-446b-8909-b128bc89c40c None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:56.049048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff2d83-acf8-446b-8909-b128bc89c40c None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:56.052736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d99d63-5d17-4796-b0c0-4a42dd13cb3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:56.053336 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d99d63-5d17-4796-b0c0-4a42dd13cb3c service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:56.053615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d99d63-5d17-4796-b0c0-4a42dd13cb3c service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:56.053885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d99d63-5d17-4796-b0c0-4a42dd13cb3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:56.062459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d99d63-5d17-4796-b0c0-4a42dd13cb3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:56.063868 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d99d63-5d17-4796-b0c0-4a42dd13cb3c service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:56.073366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 728/1446] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:19:58.705907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def7622-0bb8-4d9d-8a76-2413ea0825c4 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:58.706443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def7622-0bb8-4d9d-8a76-2413ea0825c4 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:58.706443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def7622-0bb8-4d9d-8a76-2413ea0825c4 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:58.708082 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:19:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:19:58.713416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c670a5-5e1f-4a48-aa8e-daf693f7900d None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:58.713859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c670a5-5e1f-4a48-aa8e-daf693f7900d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:58.713957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c670a5-5e1f-4a48-aa8e-daf693f7900d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:58.731029 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-93c670a5-5e1f-4a48-aa8e-daf693f7900d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:19:58.732235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-93c670a5-5e1f-4a48-aa8e-daf693f7900d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:19:58.740366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:19:58.777940 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def7622-0bb8-4d9d-8a76-2413ea0825c4 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:19:58.789866 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def7622-0bb8-4d9d-8a76-2413ea0825c4 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:19:58.793296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ed48e-a242-4d5d-ba86-5ae22dbd3d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:19:58.793925 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ed48e-a242-4d5d-ba86-5ae22dbd3d16 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:19:58.794204 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ed48e-a242-4d5d-ba86-5ae22dbd3d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:19:58.794477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ed48e-a242-4d5d-ba86-5ae22dbd3d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:19:58.803635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ed48e-a242-4d5d-ba86-5ae22dbd3d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:19:58.808361 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ed48e-a242-4d5d-ba86-5ae22dbd3d16 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:19:58.814467 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 720/1449] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:19:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:20:01.219718 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beaeeb-0920-4cc4-9c50-d53cc3aa6157 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:01.220348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beaeeb-0920-4cc4-9c50-d53cc3aa6157 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:01.220348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beaeeb-0920-4cc4-9c50-d53cc3aa6157 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:01.221694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:20:01.228073 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c04ead-5637-44e7-957a-a54ef7d4e778 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:01.228522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c04ead-5637-44e7-957a-a54ef7d4e778 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:01.228638 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c04ead-5637-44e7-957a-a54ef7d4e778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:01.246180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f4c04ead-5637-44e7-957a-a54ef7d4e778 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:20:01.247374 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f4c04ead-5637-44e7-957a-a54ef7d4e778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:01.254916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:20:01.291969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beaeeb-0920-4cc4-9c50-d53cc3aa6157 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:20:01.302859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beaeeb-0920-4cc4-9c50-d53cc3aa6157 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:20:01.306350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb315508-6472-4b1a-a6b1-8b7016461564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:20:01.306977 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb315508-6472-4b1a-a6b1-8b7016461564 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:01.307257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb315508-6472-4b1a-a6b1-8b7016461564 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:01.307531 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb315508-6472-4b1a-a6b1-8b7016461564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:01.316052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb315508-6472-4b1a-a6b1-8b7016461564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:20:01.317461 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb315508-6472-4b1a-a6b1-8b7016461564 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:20:01.326488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 731/1452] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:20:03.582919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e7708-faf8-4584-8768-16c4844448fc None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:03.583513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e7708-faf8-4584-8768-16c4844448fc None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:03.583636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e7708-faf8-4584-8768-16c4844448fc None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:03.585132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:20:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:20:03.590572 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cd845-15f6-4a45-9a7c-f7274638022d None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:03.591002 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cd845-15f6-4a45-9a7c-f7274638022d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:03.591127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cd845-15f6-4a45-9a7c-f7274638022d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:03.608641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-736cd845-15f6-4a45-9a7c-f7274638022d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:20:03.609817 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-736cd845-15f6-4a45-9a7c-f7274638022d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:03.618039 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:20:03.655725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e7708-faf8-4584-8768-16c4844448fc None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:20:03.657398 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-ac0e7708-faf8-4584-8768-16c4844448fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:03.670243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e7708-faf8-4584-8768-16c4844448fc None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:20:03.673671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b218b71-0f4d-49bd-922a-04d31db46545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:20:03.674291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b218b71-0f4d-49bd-922a-04d31db46545 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:03.674568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b218b71-0f4d-49bd-922a-04d31db46545 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:03.674840 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b218b71-0f4d-49bd-922a-04d31db46545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:03.683981 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b218b71-0f4d-49bd-922a-04d31db46545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:20:03.685374 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b218b71-0f4d-49bd-922a-04d31db46545 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:20:03.694417 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 723/1455] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:20:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:20:05.991213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ba7f-85a2-466c-bd93-c99309a2236a None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:05.991779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ba7f-85a2-466c-bd93-c99309a2236a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:05.991779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ba7f-85a2-466c-bd93-c99309a2236a None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:05.993167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:20:05.998560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f837e-d919-41bc-86bd-9a4c5fdd0fe2 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:05.998974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f837e-d919-41bc-86bd-9a4c5fdd0fe2 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:05.999133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f837e-d919-41bc-86bd-9a4c5fdd0fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:06.017064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-c33f837e-d919-41bc-86bd-9a4c5fdd0fe2 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:20:06.018245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c33f837e-d919-41bc-86bd-9a4c5fdd0fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:06.025913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:20:06.063134 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79ba7f-85a2-466c-bd93-c99309a2236a None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:20:06.074119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79ba7f-85a2-466c-bd93-c99309a2236a None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:20:06.077581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aeeb32-b85b-4f7a-9e5b-8fc2687d48ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:20:06.078208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aeeb32-b85b-4f7a-9e5b-8fc2687d48ff service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:06.078485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aeeb32-b85b-4f7a-9e5b-8fc2687d48ff service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:06.078755 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aeeb32-b85b-4f7a-9e5b-8fc2687d48ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:06.087467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aeeb32-b85b-4f7a-9e5b-8fc2687d48ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:20:06.088892 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aeeb32-b85b-4f7a-9e5b-8fc2687d48ff service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:20:06.097836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 734/1458] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:20:08.395776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867139b-f155-47f5-8e95-05ef2ae23c49 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:08.396473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867139b-f155-47f5-8e95-05ef2ae23c49 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:08.396473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867139b-f155-47f5-8e95-05ef2ae23c49 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:08.397740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:20:08.403267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb30f5-d12c-4e91-a0e4-1aed52063691 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:08.403719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb30f5-d12c-4e91-a0e4-1aed52063691 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:08.403817 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb30f5-d12c-4e91-a0e4-1aed52063691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:08.420870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-16fb30f5-d12c-4e91-a0e4-1aed52063691 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:20:08.422041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-16fb30f5-d12c-4e91-a0e4-1aed52063691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:08.430394 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:20:08.467216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867139b-f155-47f5-8e95-05ef2ae23c49 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:20:08.477719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867139b-f155-47f5-8e95-05ef2ae23c49 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:20:08.481154 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451693ff-23d3-401a-bcd2-1f0e4fc63bec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:20:08.481784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451693ff-23d3-401a-bcd2-1f0e4fc63bec service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:08.482072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451693ff-23d3-401a-bcd2-1f0e4fc63bec service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:08.482348 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451693ff-23d3-401a-bcd2-1f0e4fc63bec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:08.492117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451693ff-23d3-401a-bcd2-1f0e4fc63bec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:20:08.494448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451693ff-23d3-401a-bcd2-1f0e4fc63bec service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:20:08.508827 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 726/1461] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:20:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:20:10.800443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3ac50-c963-4be8-8b5c-b8040d434306 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:10.801023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3ac50-c963-4be8-8b5c-b8040d434306 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:10.801023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3ac50-c963-4be8-8b5c-b8040d434306 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:10.802435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:20:10.808030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28227f0-ce84-4ffd-981b-cbfb72b34f9f None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:10.808459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28227f0-ce84-4ffd-981b-cbfb72b34f9f None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:10.808569 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28227f0-ce84-4ffd-981b-cbfb72b34f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:10.826252 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d28227f0-ce84-4ffd-981b-cbfb72b34f9f None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:20:10.827416 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d28227f0-ce84-4ffd-981b-cbfb72b34f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:10.836032 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:20:10.871816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3ac50-c963-4be8-8b5c-b8040d434306 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:20:10.882543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3ac50-c963-4be8-8b5c-b8040d434306 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:20:10.885955 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef09e5a-a685-4bd8-9b36-3e1ddc29000b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:20:10.886587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef09e5a-a685-4bd8-9b36-3e1ddc29000b service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:10.886871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef09e5a-a685-4bd8-9b36-3e1ddc29000b service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:10.887138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef09e5a-a685-4bd8-9b36-3e1ddc29000b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:10.896208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef09e5a-a685-4bd8-9b36-3e1ddc29000b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:20:10.897615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef09e5a-a685-4bd8-9b36-3e1ddc29000b service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:20:10.906512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 737/1464] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:20:13.189190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee00e6-ad61-4297-9d8b-605606d70e51 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:13.190052 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee00e6-ad61-4297-9d8b-605606d70e51 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:13.190338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee00e6-ad61-4297-9d8b-605606d70e51 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:13.191706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:20:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:20:13.197373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a44636-f7a9-4091-bf88-5a4e4c5a51cb None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:13.197842 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a44636-f7a9-4091-bf88-5a4e4c5a51cb None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:13.197938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a44636-f7a9-4091-bf88-5a4e4c5a51cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:13.216366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-88a44636-f7a9-4091-bf88-5a4e4c5a51cb None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:20:13.217547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-88a44636-f7a9-4091-bf88-5a4e4c5a51cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:13.225791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:20:13.261784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee00e6-ad61-4297-9d8b-605606d70e51 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:20:13.273165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee00e6-ad61-4297-9d8b-605606d70e51 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:20:13.276619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134bf5d-b88a-436c-be5b-04bbfb42ca43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:20:13.277256 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134bf5d-b88a-436c-be5b-04bbfb42ca43 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:13.277537 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134bf5d-b88a-436c-be5b-04bbfb42ca43 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:13.277812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134bf5d-b88a-436c-be5b-04bbfb42ca43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:13.287107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134bf5d-b88a-436c-be5b-04bbfb42ca43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:20:13.288577 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134bf5d-b88a-436c-be5b-04bbfb42ca43 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:20:13.297690 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 729/1467] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:20:15.575085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4171b7-b77e-4554-891d-29f5876b4f8a None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:15.575624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4171b7-b77e-4554-891d-29f5876b4f8a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:15.575624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4171b7-b77e-4554-891d-29f5876b4f8a None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:15.577061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:20:15.582725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb6371-3f45-4a64-a3bc-200230516b08 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:15.583773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb6371-3f45-4a64-a3bc-200230516b08 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:15.583947 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb6371-3f45-4a64-a3bc-200230516b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:15.600984 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-7dfb6371-3f45-4a64-a3bc-200230516b08 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:20:15.602166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-7dfb6371-3f45-4a64-a3bc-200230516b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:15.609747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:20:15.644939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4171b7-b77e-4554-891d-29f5876b4f8a None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:20:15.655880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4171b7-b77e-4554-891d-29f5876b4f8a None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:20:15.659263 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f013a6-8a14-4e2b-a04e-1b3af7bc400a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:20:15.659913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f013a6-8a14-4e2b-a04e-1b3af7bc400a service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:15.660376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f013a6-8a14-4e2b-a04e-1b3af7bc400a service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:15.660503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f013a6-8a14-4e2b-a04e-1b3af7bc400a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:15.669218 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f013a6-8a14-4e2b-a04e-1b3af7bc400a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:20:15.670625 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f013a6-8a14-4e2b-a04e-1b3af7bc400a service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:20:15.679709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 740/1470] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:20:48.903396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635f001-e47f-4bf9-b8ba-91299b2f00fd None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:48.904051 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635f001-e47f-4bf9-b8ba-91299b2f00fd None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:48.904051 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635f001-e47f-4bf9-b8ba-91299b2f00fd None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:48.905328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 731/1471] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:20:48.910318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669da899-ac11-463e-a684-ad456e2f04a7 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:48.911289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669da899-ac11-463e-a684-ad456e2f04a7 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:48.911427 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669da899-ac11-463e-a684-ad456e2f04a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:48.928486 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-669da899-ac11-463e-a684-ad456e2f04a7 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:20:48.929669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-669da899-ac11-463e-a684-ad456e2f04a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:48.937273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 741/1472] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:20:48.983192 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b5fd0-2c9b-4b25-beed-926594497b4d None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:48.983485 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b5fd0-2c9b-4b25-beed-926594497b4d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:48.983770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b5fd0-2c9b-4b25-beed-926594497b4d None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:48.985221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 732/1473] 10.222.0.44 () {54 vars in 962 bytes} [Mon May 9 04:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:20:48.990127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3be58-1943-4b84-a559-7edcb78e1e77 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:48.990413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3be58-1943-4b84-a559-7edcb78e1e77 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:48.990691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3be58-1943-4b84-a559-7edcb78e1e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:49.007760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-ccc3be58-1943-4b84-a559-7edcb78e1e77 None None] MFA Rules not processed for user `295bb46627e043989e6dba2805607c99`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:20:49.009025 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ccc3be58-1943-4b84-a559-7edcb78e1e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:49.015119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 742/1474] 10.222.0.44 () {58 vars in 1059 bytes} [Mon May 9 04:20:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:20:49.022585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b5fd0-2c9b-4b25-beed-926594497b4d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:20:49.032870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b5fd0-2c9b-4b25-beed-926594497b4d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:20:49.037452 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b870e9f-097c-4a38-92de-c25c6cc0328c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '295bb46627e043989e6dba2805607c99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:20:49.038073 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b870e9f-097c-4a38-92de-c25c6cc0328c service glance] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:49.038359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b870e9f-097c-4a38-92de-c25c6cc0328c service glance] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:49.038637 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b870e9f-097c-4a38-92de-c25c6cc0328c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:49.047229 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b870e9f-097c-4a38-92de-c25c6cc0328c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:20:49.048672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b870e9f-097c-4a38-92de-c25c6cc0328c service glance] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:20:49.057980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 733/1475] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:20:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:20:51.375148 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b27ae-cd78-4351-883f-be893e97b3eb None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:51.375691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b27ae-cd78-4351-883f-be893e97b3eb None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:51.375691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b27ae-cd78-4351-883f-be893e97b3eb None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:51.377114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 743/1476] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:20:51.382941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fce33a-3f1d-47f8-b60a-9f780c34e6ef None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:51.383326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fce33a-3f1d-47f8-b60a-9f780c34e6ef None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:51.383502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fce33a-3f1d-47f8-b60a-9f780c34e6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:51.401123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-07fce33a-3f1d-47f8-b60a-9f780c34e6ef None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:20:51.402311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-07fce33a-3f1d-47f8-b60a-9f780c34e6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:51.409955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 734/1477] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:20:51.432594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b27ae-cd78-4351-883f-be893e97b3eb None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:20:51.445339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b27ae-cd78-4351-883f-be893e97b3eb None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:20:51.449644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d9032-10a7-41bb-816c-1fd6c0bc6e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:20:51.450311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9032-10a7-41bb-816c-1fd6c0bc6e52 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:51.450588 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9032-10a7-41bb-816c-1fd6c0bc6e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:51.450857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9032-10a7-41bb-816c-1fd6c0bc6e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:51.451578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d9032-10a7-41bb-816c-1fd6c0bc6e52 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:20:51.453368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: WARNING py.warnings [None req-8f6d9032-10a7-41bb-816c-1fd6c0bc6e52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 04:20:51.453368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: warnings.warn(msg) May 09 04:20:51.453368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: May 09 04:20:51.454483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d9032-10a7-41bb-816c-1fd6c0bc6e52 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:20:51.473315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 744/1478] 10.222.0.44 () {56 vars in 1108 bytes} [Mon May 9 04:20:51 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:20:53.736044 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942369e8-6d30-4d2c-a56b-01326ece0676 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:53.736628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942369e8-6d30-4d2c-a56b-01326ece0676 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:53.736628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942369e8-6d30-4d2c-a56b-01326ece0676 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:53.737958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 735/1479] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:20:53.750064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce62e5-0c33-41de-a109-43c19b9aca82 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:53.750064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce62e5-0c33-41de-a109-43c19b9aca82 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:53.750064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce62e5-0c33-41de-a109-43c19b9aca82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:53.763699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-77ce62e5-0c33-41de-a109-43c19b9aca82 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:20:53.764915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-77ce62e5-0c33-41de-a109-43c19b9aca82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:53.772797 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 745/1480] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:20:53.821018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942369e8-6d30-4d2c-a56b-01326ece0676 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:20:53.835079 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942369e8-6d30-4d2c-a56b-01326ece0676 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:20:53.841406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25410f7-c8c0-4bf3-abce-2bd0604a9cfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:20:53.842045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25410f7-c8c0-4bf3-abce-2bd0604a9cfb service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:53.842325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25410f7-c8c0-4bf3-abce-2bd0604a9cfb service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:53.842597 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25410f7-c8c0-4bf3-abce-2bd0604a9cfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:53.852036 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25410f7-c8c0-4bf3-abce-2bd0604a9cfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:20:53.853619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25410f7-c8c0-4bf3-abce-2bd0604a9cfb service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:20:53.867922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 736/1481] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:20:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:20:56.317893 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf616fc-bb00-48b3-b4f2-8668932f1769 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:56.318467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf616fc-bb00-48b3-b4f2-8668932f1769 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:56.318467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf616fc-bb00-48b3-b4f2-8668932f1769 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:56.319797 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 746/1482] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:20:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:20:56.324980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276809b-75b3-455a-834e-634ee3f0db0f None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:56.325469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276809b-75b3-455a-834e-634ee3f0db0f None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:56.325564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276809b-75b3-455a-834e-634ee3f0db0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:56.344458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-5276809b-75b3-455a-834e-634ee3f0db0f None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:20:56.345650 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5276809b-75b3-455a-834e-634ee3f0db0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:56.353405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 737/1483] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:20:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:20:56.389584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf616fc-bb00-48b3-b4f2-8668932f1769 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:20:56.399954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf616fc-bb00-48b3-b4f2-8668932f1769 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:20:56.403448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5a1a5-c164-4660-8b67-b43ae6ed94f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:20:56.404109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5a1a5-c164-4660-8b67-b43ae6ed94f0 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:56.404392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5a1a5-c164-4660-8b67-b43ae6ed94f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:56.404665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5a1a5-c164-4660-8b67-b43ae6ed94f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:56.413090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5a1a5-c164-4660-8b67-b43ae6ed94f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:20:56.414507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5a1a5-c164-4660-8b67-b43ae6ed94f0 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:20:56.423453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 747/1484] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:20:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:20:58.674562 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b725785-cde2-4f57-9221-60286a77691d None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:58.675132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b725785-cde2-4f57-9221-60286a77691d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:58.675132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b725785-cde2-4f57-9221-60286a77691d None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:58.677127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 738/1485] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:20:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:20:58.682375 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e386b7-784f-4849-8e16-3d6f440513a6 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:58.682827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e386b7-784f-4849-8e16-3d6f440513a6 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:58.682949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e386b7-784f-4849-8e16-3d6f440513a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:58.700599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-d7e386b7-784f-4849-8e16-3d6f440513a6 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:20:58.701773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d7e386b7-784f-4849-8e16-3d6f440513a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:20:58.709752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 748/1486] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:20:58.746991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b725785-cde2-4f57-9221-60286a77691d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:20:58.758030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b725785-cde2-4f57-9221-60286a77691d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:20:58.761800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c33903-7a19-4928-8e1a-ef386d2cc4d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:20:58.762446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c33903-7a19-4928-8e1a-ef386d2cc4d6 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:20:58.762726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c33903-7a19-4928-8e1a-ef386d2cc4d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:20:58.762993 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c33903-7a19-4928-8e1a-ef386d2cc4d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:20:58.772588 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c33903-7a19-4928-8e1a-ef386d2cc4d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:20:58.773991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c33903-7a19-4928-8e1a-ef386d2cc4d6 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:20:58.783056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 739/1487] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:20:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:21:01.150026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35d4a5-30b5-4aa8-80c6-54a040a127c5 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:21:01.150650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35d4a5-30b5-4aa8-80c6-54a040a127c5 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:21:01.150650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35d4a5-30b5-4aa8-80c6-54a040a127c5 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:21:01.151851 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 749/1488] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:21:01.157096 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f86d74-1d99-40ca-96ba-9f28f572d846 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:21:01.157533 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f86d74-1d99-40ca-96ba-9f28f572d846 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:21:01.157645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f86d74-1d99-40ca-96ba-9f28f572d846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:21:01.174675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-69f86d74-1d99-40ca-96ba-9f28f572d846 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:21:01.175919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-69f86d74-1d99-40ca-96ba-9f28f572d846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:21:01.183779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 740/1489] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:21:01.223613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35d4a5-30b5-4aa8-80c6-54a040a127c5 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:21:01.234079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35d4a5-30b5-4aa8-80c6-54a040a127c5 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:21:01.237544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12707fb3-63f9-4488-b4b0-2b906c580117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:21:01.238172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12707fb3-63f9-4488-b4b0-2b906c580117 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:21:01.238591 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12707fb3-63f9-4488-b4b0-2b906c580117 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:21:01.238864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12707fb3-63f9-4488-b4b0-2b906c580117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:21:01.247372 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12707fb3-63f9-4488-b4b0-2b906c580117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:21:01.248805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12707fb3-63f9-4488-b4b0-2b906c580117 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:21:01.258590 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 750/1490] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:21:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:21:03.769640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74955bd9-a6c7-4516-917f-c61e6f209cae None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:21:03.770236 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74955bd9-a6c7-4516-917f-c61e6f209cae None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:21:03.770236 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74955bd9-a6c7-4516-917f-c61e6f209cae None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:21:03.771590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 741/1491] 10.222.0.44 () {54 vars in 926 bytes} [Mon May 9 04:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:21:03.776874 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4991ae2-21a8-4a90-8def-494cb4387953 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:21:03.777311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4991ae2-21a8-4a90-8def-494cb4387953 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:21:03.777461 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4991ae2-21a8-4a90-8def-494cb4387953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:21:03.797169 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e4991ae2-21a8-4a90-8def-494cb4387953 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:21:03.798738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e4991ae2-21a8-4a90-8def-494cb4387953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:21:03.808969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 751/1492] 10.222.0.44 () {58 vars in 1023 bytes} [Mon May 9 04:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:21:03.847419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74955bd9-a6c7-4516-917f-c61e6f209cae None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:21:03.857932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74955bd9-a6c7-4516-917f-c61e6f209cae None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:21:03.861338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2c1ad-e934-4f60-be94-e5e3bb663ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:21:03.861966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2c1ad-e934-4f60-be94-e5e3bb663ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:21:03.862243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2c1ad-e934-4f60-be94-e5e3bb663ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:21:03.862514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2c1ad-e934-4f60-be94-e5e3bb663ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:21:03.872466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2c1ad-e934-4f60-be94-e5e3bb663ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:21:03.873894 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2c1ad-e934-4f60-be94-e5e3bb663ac5 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:21:03.884068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 742/1493] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:22:33.207588 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcac514-f086-42e2-9b86-42fa3d038e13 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.208480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcac514-f086-42e2-9b86-42fa3d038e13 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.208480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcac514-f086-42e2-9b86-42fa3d038e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.225993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-6bcac514-f086-42e2-9b86-42fa3d038e13 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:22:33.227157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6bcac514-f086-42e2-9b86-42fa3d038e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:22:33.234698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 752/1494] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:22:33.243997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ef4d7-64f6-47e4-9c3b-9e04c618c5c9 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.244506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ef4d7-64f6-47e4-9c3b-9e04c618c5c9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.244629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ef4d7-64f6-47e4-9c3b-9e04c618c5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.261869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-c52ef4d7-64f6-47e4-9c3b-9e04c618c5c9 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:22:33.263062 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c52ef4d7-64f6-47e4-9c3b-9e04c618c5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:22:33.271199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 743/1495] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:22:33.275670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcac514-f086-42e2-9b86-42fa3d038e13 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:22:33.285676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcac514-f086-42e2-9b86-42fa3d038e13 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:22:33.290232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33cf18-a820-4acf-997f-5c71a2cedd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:22:33.290877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33cf18-a820-4acf-997f-5c71a2cedd06 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.291156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33cf18-a820-4acf-997f-5c71a2cedd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.291430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33cf18-a820-4acf-997f-5c71a2cedd06 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.292207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33cf18-a820-4acf-997f-5c71a2cedd06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:22:33.292504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33cf18-a820-4acf-997f-5c71a2cedd06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:22:33.293500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: WARNING py.warnings [None req-4e33cf18-a820-4acf-997f-5c71a2cedd06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 04:22:33.293500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: warnings.warn(msg) May 09 04:22:33.293500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: May 09 04:22:33.294409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33cf18-a820-4acf-997f-5c71a2cedd06 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:22:33.305529 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 753/1496] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:22:33.313464 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ef4d7-64f6-47e4-9c3b-9e04c618c5c9 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:22:33.322920 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ef4d7-64f6-47e4-9c3b-9e04c618c5c9 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:22:33.327428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343da5b2-2be6-4396-8ac5-20571e1ca9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:22:33.328170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343da5b2-2be6-4396-8ac5-20571e1ca9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.328455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343da5b2-2be6-4396-8ac5-20571e1ca9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.328732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343da5b2-2be6-4396-8ac5-20571e1ca9b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.329410 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343da5b2-2be6-4396-8ac5-20571e1ca9b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:22:33.331736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: WARNING py.warnings [None req-343da5b2-2be6-4396-8ac5-20571e1ca9b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 04:22:33.331736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: warnings.warn(msg) May 09 04:22:33.331736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: May 09 04:22:33.332940 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343da5b2-2be6-4396-8ac5-20571e1ca9b6 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:22:33.367113 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 744/1497] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:22:33 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:22:33.371452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33cf18-a820-4acf-997f-5c71a2cedd06 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:22:33.381598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33cf18-a820-4acf-997f-5c71a2cedd06 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:22:33.433561 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe1a22-7ff3-4498-b78d-972299f35442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:22:33.434214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe1a22-7ff3-4498-b78d-972299f35442 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.434493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe1a22-7ff3-4498-b78d-972299f35442 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.434764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe1a22-7ff3-4498-b78d-972299f35442 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.435457 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe1a22-7ff3-4498-b78d-972299f35442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:22:33.436470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: WARNING py.warnings [None req-25fe1a22-7ff3-4498-b78d-972299f35442 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 04:22:33.436470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: warnings.warn(msg) May 09 04:22:33.436470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: May 09 04:22:33.437650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe1a22-7ff3-4498-b78d-972299f35442 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:22:33.459987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 754/1498] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:22:33 2022] POST /identity/v3/users => generated 290 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:22:33.463425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343da5b2-2be6-4396-8ac5-20571e1ca9b6 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:22:33.473396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343da5b2-2be6-4396-8ac5-20571e1ca9b6 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:22:33.477790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48ee80-87d5-45fa-8ce0-ce1af18e4074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:22:33.478448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48ee80-87d5-45fa-8ce0-ce1af18e4074 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.478731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48ee80-87d5-45fa-8ce0-ce1af18e4074 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.479007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48ee80-87d5-45fa-8ce0-ce1af18e4074 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.479729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48ee80-87d5-45fa-8ce0-ce1af18e4074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:22:33.480770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: WARNING py.warnings [None req-0c48ee80-87d5-45fa-8ce0-ce1af18e4074 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 04:22:33.480770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: warnings.warn(msg) May 09 04:22:33.480770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: May 09 04:22:33.481633 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48ee80-87d5-45fa-8ce0-ce1af18e4074 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:22:33.494120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 745/1499] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:22:33.498102 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe1a22-7ff3-4498-b78d-972299f35442 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:22:33.508534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe1a22-7ff3-4498-b78d-972299f35442 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:22:33.513082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671656dd-aac3-4844-952c-3110ff682608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:22:33.513753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671656dd-aac3-4844-952c-3110ff682608 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.514038 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671656dd-aac3-4844-952c-3110ff682608 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.514317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671656dd-aac3-4844-952c-3110ff682608 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.515039 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671656dd-aac3-4844-952c-3110ff682608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:22:33.516120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: WARNING py.warnings [None req-671656dd-aac3-4844-952c-3110ff682608 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 04:22:33.516120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: warnings.warn(msg) May 09 04:22:33.516120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: May 09 04:22:33.516994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671656dd-aac3-4844-952c-3110ff682608 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:22:33.530787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 755/1500] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:22:33.534801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48ee80-87d5-45fa-8ce0-ce1af18e4074 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:22:33.544279 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48ee80-87d5-45fa-8ce0-ce1af18e4074 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:22:33.548902 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ab24f-9d7d-4170-847a-b6803b01e028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:22:33.550014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ab24f-9d7d-4170-847a-b6803b01e028 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.550296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ab24f-9d7d-4170-847a-b6803b01e028 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.550573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ab24f-9d7d-4170-847a-b6803b01e028 admin admin] PATH_INFO: `/v3/projects/e73bb050c7134619a8df0308b4649d6d/users/6d0a1bed7776416cb431420977fb13b6/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.565874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ab24f-9d7d-4170-847a-b6803b01e028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e73bb050c7134619a8df0308b4649d6d, user_id=6d0a1bed7776416cb431420977fb13b6, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:22:33.566902 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: WARNING py.warnings [None req-839ab24f-9d7d-4170-847a-b6803b01e028 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 04:22:33.566902 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: warnings.warn(msg) May 09 04:22:33.566902 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: May 09 04:22:33.568845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ab24f-9d7d-4170-847a-b6803b01e028 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:22:33.581952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 746/1501] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:22:33 2022] PUT /identity/v3/projects/e73bb050c7134619a8df0308b4649d6d/users/6d0a1bed7776416cb431420977fb13b6/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:22:33.593288 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfd598-321d-46c9-ae8c-98852a8c7dbc None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.593582 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfd598-321d-46c9-ae8c-98852a8c7dbc None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.593863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfd598-321d-46c9-ae8c-98852a8c7dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.619289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e8dfd598-321d-46c9-ae8c-98852a8c7dbc None None] MFA Rules not processed for user `6d0a1bed7776416cb431420977fb13b6`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:22:33.620512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e8dfd598-321d-46c9-ae8c-98852a8c7dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:22:33.683085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 756/1502] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:22:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:22:33.697149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32757261-1bf8-4eec-a167-df54e84ecee5 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.697443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32757261-1bf8-4eec-a167-df54e84ecee5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.697726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32757261-1bf8-4eec-a167-df54e84ecee5 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.699086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 747/1503] 10.222.0.44 () {54 vars in 963 bytes} [Mon May 9 04:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:22:33.703634 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a7ca0-29c1-4533-bd9a-74b1bde0f414 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.703947 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a7ca0-29c1-4533-bd9a-74b1bde0f414 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.704224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a7ca0-29c1-4533-bd9a-74b1bde0f414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.722909 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-512a7ca0-29c1-4533-bd9a-74b1bde0f414 None None] MFA Rules not processed for user `72caf90f411f495bb22aa85df4d25eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:22:33.724110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-512a7ca0-29c1-4533-bd9a-74b1bde0f414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:22:33.773642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 757/1504] 10.222.0.44 () {58 vars in 1060 bytes} [Mon May 9 04:22:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:22:33.781143 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32757261-1bf8-4eec-a167-df54e84ecee5 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:22:33.791527 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32757261-1bf8-4eec-a167-df54e84ecee5 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:22:33.795612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42108d-aaba-4454-b56a-66494a7c3320 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:22:33.796569 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42108d-aaba-4454-b56a-66494a7c3320 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.796908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42108d-aaba-4454-b56a-66494a7c3320 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.797314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42108d-aaba-4454-b56a-66494a7c3320 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.808915 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42108d-aaba-4454-b56a-66494a7c3320 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:22:33.810358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42108d-aaba-4454-b56a-66494a7c3320 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:22:33.819696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 748/1505] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:22:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:22:33.861794 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed53e8-8b38-4ba7-90e2-29d6c82e668d None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.862186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed53e8-8b38-4ba7-90e2-29d6c82e668d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.862463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed53e8-8b38-4ba7-90e2-29d6c82e668d None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.863743 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 758/1506] 10.222.0.44 () {52 vars in 829 bytes} [Mon May 9 04:22:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:22:33.867705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42108d-aaba-4454-b56a-66494a7c3320 service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:22:33.926427 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42108d-aaba-4454-b56a-66494a7c3320 service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:22:33.930843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:22:33.931704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:33.932019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:33.932298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] PATH_INFO: `/v3/users/6d0a1bed7776416cb431420977fb13b6` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:33.933462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d0a1bed7776416cb431420977fb13b6)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:22:33.934465 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: WARNING py.warnings [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 04:22:33.934465 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: warnings.warn(msg) May 09 04:22:33.934465 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: May 09 04:22:33.935659 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:22:33.997417 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d0a1bed7776416cb431420977fb13b6'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:22:33.997982 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d0a1bed7776416cb431420977fb13b6'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:22:33.998684 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d0a1bed7776416cb431420977fb13b6'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:22:34.008591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d0a1bed7776416cb431420977fb13b6'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:22:34.020001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d0a1bed7776416cb431420977fb13b6'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:22:34.042210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 749/1507] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:22:33 2022] DELETE /identity/v3/users/6d0a1bed7776416cb431420977fb13b6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:22:34.046710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e5e0-f5f0-4cfb-9991-1baf79498d9f None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:34.047001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e5e0-f5f0-4cfb-9991-1baf79498d9f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:34.047283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e5e0-f5f0-4cfb-9991-1baf79498d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:34.064734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-6bd9e5e0-f5f0-4cfb-9991-1baf79498d9f None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:22:34.065988 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6bd9e5e0-f5f0-4cfb-9991-1baf79498d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:22:34.121096 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 759/1508] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:22:34.130952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:22:34.132403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:22:34.186971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708ef9a-35eb-41bc-bd01-86321cebcb69 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:22:34.190733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd554b-7067-4c96-adbd-ce6a674730c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:22:34.191373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd554b-7067-4c96-adbd-ce6a674730c3 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:34.191659 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd554b-7067-4c96-adbd-ce6a674730c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:34.191954 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd554b-7067-4c96-adbd-ce6a674730c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:34.200546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd554b-7067-4c96-adbd-ce6a674730c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:22:34.201951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd554b-7067-4c96-adbd-ce6a674730c3 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:22:34.211572 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 750/1509] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:22:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:22:34.552907 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9e5e0-f5f0-4cfb-9991-1baf79498d9f None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:22:34.554343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6bd9e5e0-f5f0-4cfb-9991-1baf79498d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:22:34.566711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9e5e0-f5f0-4cfb-9991-1baf79498d9f None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:22:34.573072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5035e-39ec-4e15-82de-87c7c975dfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:22:34.574014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5035e-39ec-4e15-82de-87c7c975dfe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:22:34.574251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5035e-39ec-4e15-82de-87c7c975dfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:22:34.574529 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5035e-39ec-4e15-82de-87c7c975dfe5 admin admin] PATH_INFO: `/v3/projects/e73bb050c7134619a8df0308b4649d6d` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:22:34.575742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5035e-39ec-4e15-82de-87c7c975dfe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e73bb050c7134619a8df0308b4649d6d)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:22:34.576778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: WARNING py.warnings [None req-09e5035e-39ec-4e15-82de-87c7c975dfe5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 04:22:34.576778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: warnings.warn(msg) May 09 04:22:34.576778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: May 09 04:22:34.578747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5035e-39ec-4e15-82de-87c7c975dfe5 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:22:34.636669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-09e5035e-39ec-4e15-82de-87c7c975dfe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e73bb050c7134619a8df0308b4649d6d', 'request_id': 'req-09e5035e-39ec-4e15-82de-87c7c975dfe5'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:22:34.642289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-09e5035e-39ec-4e15-82de-87c7c975dfe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e73bb050c7134619a8df0308b4649d6d', 'request_id': 'req-09e5035e-39ec-4e15-82de-87c7c975dfe5'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:22:34.651079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-09e5035e-39ec-4e15-82de-87c7c975dfe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e73bb050c7134619a8df0308b4649d6d', 'request_id': 'req-09e5035e-39ec-4e15-82de-87c7c975dfe5'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:22:34.670070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-09e5035e-39ec-4e15-82de-87c7c975dfe5 admin admin] The token cache is being invalidate because project e73bb050c7134619a8df0308b4649d6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:22:34.670358 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-09e5035e-39ec-4e15-82de-87c7c975dfe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:22:34.671077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-09e5035e-39ec-4e15-82de-87c7c975dfe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:22:34.672656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 760/1510] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:22:34 2022] DELETE /identity/v3/projects/e73bb050c7134619a8df0308b4649d6d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:24:12.863462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668a786-63fc-42b5-bac7-1f6549ceabb3 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:12.864166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668a786-63fc-42b5-bac7-1f6549ceabb3 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:12.864166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668a786-63fc-42b5-bac7-1f6549ceabb3 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:12.866954 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 04:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:24:12.870561 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab748d6-993a-41e4-8de6-56b5d4367ff7 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:12.871053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab748d6-993a-41e4-8de6-56b5d4367ff7 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:12.871157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab748d6-993a-41e4-8de6-56b5d4367ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:12.888713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-6ab748d6-993a-41e4-8de6-56b5d4367ff7 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:24:12.889909 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6ab748d6-993a-41e4-8de6-56b5d4367ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:12.947483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 04:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:24:13.128628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668a786-63fc-42b5-bac7-1f6549ceabb3 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:24:13.130132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3668a786-63fc-42b5-bac7-1f6549ceabb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:13.203361 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668a786-63fc-42b5-bac7-1f6549ceabb3 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:24:13.209599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e7195-417d-43be-8dd6-0354fe2a8cf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:24:13.210268 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e7195-417d-43be-8dd6-0354fe2a8cf3 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:13.210551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e7195-417d-43be-8dd6-0354fe2a8cf3 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:13.210825 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e7195-417d-43be-8dd6-0354fe2a8cf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:13.218692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e7195-417d-43be-8dd6-0354fe2a8cf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:24:13.220295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e7195-417d-43be-8dd6-0354fe2a8cf3 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:24:13.229496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 752/1513] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:24:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:24:13.534008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc2fb-d274-47c4-8d29-fe86e1f7f689 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:13.534315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc2fb-d274-47c4-8d29-fe86e1f7f689 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:13.534595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc2fb-d274-47c4-8d29-fe86e1f7f689 None None] PATH_INFO: `/v3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:13.535953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 762/1514] 10.222.0.13 () {54 vars in 925 bytes} [Mon May 9 04:24:13 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:24:13.542011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c785f-df6d-4762-80a5-00fecbd91043 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:13.542311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c785f-df6d-4762-80a5-00fecbd91043 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:13.542594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c785f-df6d-4762-80a5-00fecbd91043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:13.561532 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-481c785f-df6d-4762-80a5-00fecbd91043 None None] MFA Rules not processed for user `d41e05b7ef954dd596c0134b65434dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:24:13.562771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-481c785f-df6d-4762-80a5-00fecbd91043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:13.623213 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 753/1515] 10.222.0.13 () {58 vars in 1015 bytes} [Mon May 9 04:24:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:24:14.135571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d49ba-7246-491b-b504-819c57eaaafd None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:14.136133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d49ba-7246-491b-b504-819c57eaaafd None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:14.136133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d49ba-7246-491b-b504-819c57eaaafd None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:14.137425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 763/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 04:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:24:14.142351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4d68c-a9aa-4d20-a679-0765ca56a61c None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:14.142779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4d68c-a9aa-4d20-a679-0765ca56a61c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:14.142910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4d68c-a9aa-4d20-a679-0765ca56a61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:14.162170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-90b4d68c-a9aa-4d20-a679-0765ca56a61c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:24:14.163362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-90b4d68c-a9aa-4d20-a679-0765ca56a61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:14.171399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 754/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 04:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:24:14.320790 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d49ba-7246-491b-b504-819c57eaaafd None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:24:14.335520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d49ba-7246-491b-b504-819c57eaaafd None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:24:14.340852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d757318-2af3-4e47-a1b3-35abadb02083 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:24:14.341490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d757318-2af3-4e47-a1b3-35abadb02083 service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:14.341772 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d757318-2af3-4e47-a1b3-35abadb02083 service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:14.342048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d757318-2af3-4e47-a1b3-35abadb02083 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:14.350879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d757318-2af3-4e47-a1b3-35abadb02083 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:24:14.352518 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d757318-2af3-4e47-a1b3-35abadb02083 service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:24:14.361846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 764/1518] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:24:17.418158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e090278-43aa-43a5-a34a-fb192dc8a694 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:17.418668 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e090278-43aa-43a5-a34a-fb192dc8a694 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:17.418668 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e090278-43aa-43a5-a34a-fb192dc8a694 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:17.419996 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 755/1519] 10.222.0.44 () {54 vars in 960 bytes} [Mon May 9 04:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:24:17.425524 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e15e0d-f2a6-44bf-86f8-f47127af03d2 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:17.425950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e15e0d-f2a6-44bf-86f8-f47127af03d2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:17.426059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e15e0d-f2a6-44bf-86f8-f47127af03d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:17.443970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-b0e15e0d-f2a6-44bf-86f8-f47127af03d2 None None] MFA Rules not processed for user `d41e05b7ef954dd596c0134b65434dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:24:17.445144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b0e15e0d-f2a6-44bf-86f8-f47127af03d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:17.453444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 765/1520] 10.222.0.44 () {58 vars in 1057 bytes} [Mon May 9 04:24:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:24:17.460951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e090278-43aa-43a5-a34a-fb192dc8a694 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:24:17.473629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e090278-43aa-43a5-a34a-fb192dc8a694 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:24:17.479723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e24da3-b058-4d29-9188-dc59d06757c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd41e05b7ef954dd596c0134b65434dd6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:24:17.480408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e24da3-b058-4d29-9188-dc59d06757c8 service placement] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:17.480773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e24da3-b058-4d29-9188-dc59d06757c8 service placement] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:17.481056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e24da3-b058-4d29-9188-dc59d06757c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:17.491974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e24da3-b058-4d29-9188-dc59d06757c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:24:17.493663 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e24da3-b058-4d29-9188-dc59d06757c8 service placement] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:24:17.504243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 756/1521] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:24:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:24:25.674431 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d101-b442-4752-8465-a2565ac675d4 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:25.674853 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d101-b442-4752-8465-a2565ac675d4 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:25.675000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d101-b442-4752-8465-a2565ac675d4 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:25.676428 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 04:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:24:25.681327 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76496c4a-ff78-4280-bc19-244529fd8977 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:25.681764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76496c4a-ff78-4280-bc19-244529fd8977 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:25.681884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76496c4a-ff78-4280-bc19-244529fd8977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:25.699277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-76496c4a-ff78-4280-bc19-244529fd8977 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:24:25.700513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-76496c4a-ff78-4280-bc19-244529fd8977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:25.708846 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 04:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:24:25.859362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51d101-b442-4752-8465-a2565ac675d4 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:24:25.873956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51d101-b442-4752-8465-a2565ac675d4 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:24:25.880244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc268d9-a05f-433e-9b59-b96e8dae18a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:24:25.880886 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc268d9-a05f-433e-9b59-b96e8dae18a2 service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:25.881187 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc268d9-a05f-433e-9b59-b96e8dae18a2 service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:25.881460 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc268d9-a05f-433e-9b59-b96e8dae18a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:25.889520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc268d9-a05f-433e-9b59-b96e8dae18a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:24:25.891079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc268d9-a05f-433e-9b59-b96e8dae18a2 service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:24:25.900929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 767/1524] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:24:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:24:26.794136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48342803-caba-4958-b07f-e12b99aa53e9 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:26.794630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48342803-caba-4958-b07f-e12b99aa53e9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:26.794727 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48342803-caba-4958-b07f-e12b99aa53e9 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:26.796142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 9 04:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:24:26.801094 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07ec7f-1308-4ec6-b6e4-6b0f0ee5db54 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:26.801528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07ec7f-1308-4ec6-b6e4-6b0f0ee5db54 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:26.801638 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07ec7f-1308-4ec6-b6e4-6b0f0ee5db54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:26.819736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-9f07ec7f-1308-4ec6-b6e4-6b0f0ee5db54 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:24:26.820953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-9f07ec7f-1308-4ec6-b6e4-6b0f0ee5db54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:26.828691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 9 04:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:24:26.978219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48342803-caba-4958-b07f-e12b99aa53e9 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:24:26.993157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48342803-caba-4958-b07f-e12b99aa53e9 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:24:26.998387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876e589-5227-47bc-bde2-afe8ac91fc59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:24:26.999014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876e589-5227-47bc-bde2-afe8ac91fc59 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:26.999295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876e589-5227-47bc-bde2-afe8ac91fc59 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:26.999570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876e589-5227-47bc-bde2-afe8ac91fc59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:27.008196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876e589-5227-47bc-bde2-afe8ac91fc59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:24:27.009800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876e589-5227-47bc-bde2-afe8ac91fc59 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:24:27.018695 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 759/1527] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:24:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:24:31.165416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe66817-cd22-48e3-a253-bf0f0df98b58 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:31.165949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe66817-cd22-48e3-a253-bf0f0df98b58 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:31.165949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe66817-cd22-48e3-a253-bf0f0df98b58 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:31.167245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:24:31.172380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b29d4-297f-4142-b3b8-c0d3efb0851e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:31.172787 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b29d4-297f-4142-b3b8-c0d3efb0851e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:31.172943 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4b29d4-297f-4142-b3b8-c0d3efb0851e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:31.191190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-2b4b29d4-297f-4142-b3b8-c0d3efb0851e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:24:31.192408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-2b4b29d4-297f-4142-b3b8-c0d3efb0851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:31.199993 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:24:31.236264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe66817-cd22-48e3-a253-bf0f0df98b58 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:24:31.238024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ffe66817-cd22-48e3-a253-bf0f0df98b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:31.291747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe66817-cd22-48e3-a253-bf0f0df98b58 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:24:31.295161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad76474-658f-4836-9a31-85e60a5114d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:24:31.295788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad76474-658f-4836-9a31-85e60a5114d5 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:31.296095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad76474-658f-4836-9a31-85e60a5114d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:31.296368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad76474-658f-4836-9a31-85e60a5114d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:31.304958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad76474-658f-4836-9a31-85e60a5114d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:24:31.306356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad76474-658f-4836-9a31-85e60a5114d5 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:24:31.315789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 770/1530] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:24:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:24:33.519748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aec09-604c-40bb-9814-995703d2c08d None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:33.520384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aec09-604c-40bb-9814-995703d2c08d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:33.520384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aec09-604c-40bb-9814-995703d2c08d None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:33.521728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:24:33.527069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518a3a2-bd3d-4f92-a228-0a574ef9cf23 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:33.527494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518a3a2-bd3d-4f92-a228-0a574ef9cf23 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:33.527631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518a3a2-bd3d-4f92-a228-0a574ef9cf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:33.545873 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-5518a3a2-bd3d-4f92-a228-0a574ef9cf23 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:24:33.547097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5518a3a2-bd3d-4f92-a228-0a574ef9cf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:33.555619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:24:33.590290 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aec09-604c-40bb-9814-995703d2c08d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:24:33.602257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aec09-604c-40bb-9814-995703d2c08d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:24:33.606208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9602f5c-ff5d-41ea-88df-01b2c9444955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:24:33.606944 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9602f5c-ff5d-41ea-88df-01b2c9444955 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:33.607326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9602f5c-ff5d-41ea-88df-01b2c9444955 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:33.607755 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9602f5c-ff5d-41ea-88df-01b2c9444955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:33.617452 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9602f5c-ff5d-41ea-88df-01b2c9444955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:24:33.618863 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9602f5c-ff5d-41ea-88df-01b2c9444955 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:24:33.628011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 762/1533] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:24:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:24:37.219719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f0b10-678e-423d-adec-a4c6128f5262 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:37.220352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f0b10-678e-423d-adec-a4c6128f5262 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:37.220352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f0b10-678e-423d-adec-a4c6128f5262 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:37.222209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:24:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:24:37.227558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b1f0c-93ea-45ff-8384-61ae5cf68130 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:37.228023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b1f0c-93ea-45ff-8384-61ae5cf68130 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:37.228136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b1f0c-93ea-45ff-8384-61ae5cf68130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:37.246255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d95b1f0c-93ea-45ff-8384-61ae5cf68130 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:24:37.247440 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d95b1f0c-93ea-45ff-8384-61ae5cf68130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:37.255104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:24:37.531462 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f0b10-678e-423d-adec-a4c6128f5262 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:24:37.533135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3b2f0b10-678e-423d-adec-a4c6128f5262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:37.588762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f0b10-678e-423d-adec-a4c6128f5262 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:24:37.592172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc3c51-6b22-4e79-ac3d-58b6248d947b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:24:37.592804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc3c51-6b22-4e79-ac3d-58b6248d947b service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:37.593080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc3c51-6b22-4e79-ac3d-58b6248d947b service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:37.593348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc3c51-6b22-4e79-ac3d-58b6248d947b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:37.601813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc3c51-6b22-4e79-ac3d-58b6248d947b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:24:37.603210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc3c51-6b22-4e79-ac3d-58b6248d947b service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:24:37.612755 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 773/1536] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:24:39.887460 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ebad1-e298-4f42-88bc-9b3819b2875d None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:39.888056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ebad1-e298-4f42-88bc-9b3819b2875d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:39.888056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ebad1-e298-4f42-88bc-9b3819b2875d None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:39.889397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 9 04:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:24:39.895206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7bdac-b83b-46c4-8d83-5b969f8084b7 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:39.895646 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7bdac-b83b-46c4-8d83-5b969f8084b7 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:39.895757 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7bdac-b83b-46c4-8d83-5b969f8084b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:39.913629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-5db7bdac-b83b-46c4-8d83-5b969f8084b7 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:24:39.914802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5db7bdac-b83b-46c4-8d83-5b969f8084b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:39.922735 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 9 04:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:24:40.193646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ebad1-e298-4f42-88bc-9b3819b2875d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:24:40.195303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b53ebad1-e298-4f42-88bc-9b3819b2875d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:24:40.208606 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ebad1-e298-4f42-88bc-9b3819b2875d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:24:40.212297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09f3ca-1f2e-4074-b7bc-1a2b61b56044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:24:40.212928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09f3ca-1f2e-4074-b7bc-1a2b61b56044 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:24:40.213210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09f3ca-1f2e-4074-b7bc-1a2b61b56044 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:24:40.213486 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09f3ca-1f2e-4074-b7bc-1a2b61b56044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:24:40.222136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09f3ca-1f2e-4074-b7bc-1a2b61b56044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:24:40.223537 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09f3ca-1f2e-4074-b7bc-1a2b61b56044 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:24:40.232583 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 765/1539] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:10.467694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be759ce0-b489-4b44-82ec-90f253044421 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.468415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be759ce0-b489-4b44-82ec-90f253044421 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.468415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be759ce0-b489-4b44-82ec-90f253044421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.494360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-be759ce0-b489-4b44-82ec-90f253044421 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:10.495569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-be759ce0-b489-4b44-82ec-90f253044421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:10.506046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 775/1540] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:10.516667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf03373-216f-442d-b369-196e64b0853c None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.517717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf03373-216f-442d-b369-196e64b0853c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.518216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf03373-216f-442d-b369-196e64b0853c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.542049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-aaf03373-216f-442d-b369-196e64b0853c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:10.543604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-aaf03373-216f-442d-b369-196e64b0853c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:10.556363 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a73ad-9ba1-47ae-ad7a-aa22cdba3417 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.556765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 766/1541] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:10.557264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a73ad-9ba1-47ae-ad7a-aa22cdba3417 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.557817 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a73ad-9ba1-47ae-ad7a-aa22cdba3417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.561704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf03373-216f-442d-b369-196e64b0853c None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:10.572088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf03373-216f-442d-b369-196e64b0853c None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:10.579643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996325cb-1525-48f9-a231-9b40d02e756c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:10.580349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996325cb-1525-48f9-a231-9b40d02e756c admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.580757 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996325cb-1525-48f9-a231-9b40d02e756c admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.581039 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996325cb-1525-48f9-a231-9b40d02e756c admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.581771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996325cb-1525-48f9-a231-9b40d02e756c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:10.582077 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996325cb-1525-48f9-a231-9b40d02e756c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:10.587916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: WARNING py.warnings [None req-996325cb-1525-48f9-a231-9b40d02e756c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 09 04:26:10.587916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: warnings.warn(msg) May 09 04:26:10.587916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: May 09 04:26:10.587916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996325cb-1525-48f9-a231-9b40d02e756c admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:10.589986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2e4a73ad-9ba1-47ae-ad7a-aa22cdba3417 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:10.591453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2e4a73ad-9ba1-47ae-ad7a-aa22cdba3417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:10.602761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 767/1542] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:10.609819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 776/1543] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:26:10.610721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996325cb-1525-48f9-a231-9b40d02e756c admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:10.620945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786204a0-0194-4dda-bd22-a1650048f4d6 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.622257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786204a0-0194-4dda-bd22-a1650048f4d6 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.622755 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786204a0-0194-4dda-bd22-a1650048f4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.631807 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996325cb-1525-48f9-a231-9b40d02e756c admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:10.640544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae6e2b-730f-447b-b794-1f0bd6234341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:10.641215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae6e2b-730f-447b-b794-1f0bd6234341 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.641501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae6e2b-730f-447b-b794-1f0bd6234341 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.641896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae6e2b-730f-447b-b794-1f0bd6234341 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.642593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae6e2b-730f-447b-b794-1f0bd6234341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:10.644533 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae6e2b-730f-447b-b794-1f0bd6234341 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:10.661020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-786204a0-0194-4dda-bd22-a1650048f4d6 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:10.662836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-786204a0-0194-4dda-bd22-a1650048f4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:10.678621 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 777/1544] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:26:10.688061 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 768/1545] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/projects => generated 377 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:26:10.692678 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f954078-fe7c-4b22-b638-0e5b0cedc85f None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.692915 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae6e2b-730f-447b-b794-1f0bd6234341 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:10.693039 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f954078-fe7c-4b22-b638-0e5b0cedc85f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.693884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f954078-fe7c-4b22-b638-0e5b0cedc85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.721256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-9f954078-fe7c-4b22-b638-0e5b0cedc85f None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:10.722352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-9f954078-fe7c-4b22-b638-0e5b0cedc85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:10.766630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae6e2b-730f-447b-b794-1f0bd6234341 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:10.771089 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a273765-32cc-4310-b6b0-c63432258edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:10.771785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a273765-32cc-4310-b6b0-c63432258edd admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.772092 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a273765-32cc-4310-b6b0-c63432258edd admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.772368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a273765-32cc-4310-b6b0-c63432258edd admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.773083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a273765-32cc-4310-b6b0-c63432258edd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:10.773499 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a273765-32cc-4310-b6b0-c63432258edd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:10.775103 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a273765-32cc-4310-b6b0-c63432258edd admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:10.780883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 778/1546] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:10.782099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef331b-aa59-41b3-b572-41e693ad4fdb None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.782389 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef331b-aa59-41b3-b572-41e693ad4fdb None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.782664 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef331b-aa59-41b3-b572-41e693ad4fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.786489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 769/1547] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:10.787171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a273765-32cc-4310-b6b0-c63432258edd admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:10.796757 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a273765-32cc-4310-b6b0-c63432258edd admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:10.801291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adb2af-113a-46f1-a0b9-08d36396d4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:10.801936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adb2af-113a-46f1-a0b9-08d36396d4ad admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.802220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adb2af-113a-46f1-a0b9-08d36396d4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.802493 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adb2af-113a-46f1-a0b9-08d36396d4ad admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.803716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20adb2af-113a-46f1-a0b9-08d36396d4ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:10.804748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: WARNING py.warnings [None req-20adb2af-113a-46f1-a0b9-08d36396d4ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 04:26:10.804748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: warnings.warn(msg) May 09 04:26:10.804748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: May 09 04:26:10.805945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20adb2af-113a-46f1-a0b9-08d36396d4ad admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:10.806354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-4cef331b-aa59-41b3-b572-41e693ad4fdb None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:10.807528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4cef331b-aa59-41b3-b572-41e693ad4fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:10.816727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 779/1548] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:10.817892 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d6443-259f-4cbe-a2df-5d06efe3df8c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.818864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d6443-259f-4cbe-a2df-5d06efe3df8c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.819159 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d6443-259f-4cbe-a2df-5d06efe3df8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.837429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 770/1549] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:10.837429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8186a1-d4a7-41c3-bbd8-d2ab4e5a97f8 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.837429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8186a1-d4a7-41c3-bbd8-d2ab4e5a97f8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.837429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8186a1-d4a7-41c3-bbd8-d2ab4e5a97f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.837876 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-529d6443-259f-4cbe-a2df-5d06efe3df8c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:10.838566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-529d6443-259f-4cbe-a2df-5d06efe3df8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:10.846429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 780/1550] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:10.847643 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff1980-4460-46f9-9fb5-9bcea48a29d2 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.847956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff1980-4460-46f9-9fb5-9bcea48a29d2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.848567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff1980-4460-46f9-9fb5-9bcea48a29d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.855028 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-9b8186a1-d4a7-41c3-bbd8-d2ab4e5a97f8 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:10.856264 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9b8186a1-d4a7-41c3-bbd8-d2ab4e5a97f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:10.863969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 771/1551] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:10.866587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-5bff1980-4460-46f9-9fb5-9bcea48a29d2 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:10.867826 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5bff1980-4460-46f9-9fb5-9bcea48a29d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:10.868370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b946b-54cd-48ce-af5b-3cec1647ca26 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.868824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b946b-54cd-48ce-af5b-3cec1647ca26 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.869260 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b946b-54cd-48ce-af5b-3cec1647ca26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.876748 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 781/1552] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:10.877922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b21fe7-fd09-46a1-8d27-9f1f346f6d15 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.878211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b21fe7-fd09-46a1-8d27-9f1f346f6d15 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.878488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b21fe7-fd09-46a1-8d27-9f1f346f6d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.888912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-053b946b-54cd-48ce-af5b-3cec1647ca26 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:10.890118 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-053b946b-54cd-48ce-af5b-3cec1647ca26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:10.895994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-04b21fe7-fd09-46a1-8d27-9f1f346f6d15 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:10.897640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-04b21fe7-fd09-46a1-8d27-9f1f346f6d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:10.898477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 772/1553] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:10.899366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b946b-54cd-48ce-af5b-3cec1647ca26 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:10.909034 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 782/1554] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:10.909034 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd5c43-4d4b-47de-a791-495b6aa7ac56 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.909034 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd5c43-4d4b-47de-a791-495b6aa7ac56 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.909034 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd5c43-4d4b-47de-a791-495b6aa7ac56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.912431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b946b-54cd-48ce-af5b-3cec1647ca26 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:10.917071 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559347a2-1c99-4675-aa99-a504cefb1aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:10.917771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559347a2-1c99-4675-aa99-a504cefb1aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.918053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559347a2-1c99-4675-aa99-a504cefb1aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.918326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559347a2-1c99-4675-aa99-a504cefb1aa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.919005 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559347a2-1c99-4675-aa99-a504cefb1aa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:10.920910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559347a2-1c99-4675-aa99-a504cefb1aa4 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:10.924780 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-6edd5c43-4d4b-47de-a791-495b6aa7ac56 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:10.926065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6edd5c43-4d4b-47de-a791-495b6aa7ac56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:10.935151 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 783/1555] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:10.935994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd5c43-4d4b-47de-a791-495b6aa7ac56 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:10.946355 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd5c43-4d4b-47de-a791-495b6aa7ac56 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:10.951111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 773/1556] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:10.951111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382408de-ba30-4d11-b340-59c3bb5fd244 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:10.951111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382408de-ba30-4d11-b340-59c3bb5fd244 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:10.951111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382408de-ba30-4d11-b340-59c3bb5fd244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:10.968728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-382408de-ba30-4d11-b340-59c3bb5fd244 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:10.970053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-382408de-ba30-4d11-b340-59c3bb5fd244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:11.000353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17b610-f576-448d-b1a4-338f5bbcdf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.000945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17b610-f576-448d-b1a4-338f5bbcdf7d admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.001224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17b610-f576-448d-b1a4-338f5bbcdf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.001505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17b610-f576-448d-b1a4-338f5bbcdf7d admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.002222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17b610-f576-448d-b1a4-338f5bbcdf7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.003966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17b610-f576-448d-b1a4-338f5bbcdf7d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.014463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 784/1557] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:11.015693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e969169-6778-417c-8ccd-f8852a2d79ec None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.016003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e969169-6778-417c-8ccd-f8852a2d79ec None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.016281 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e969169-6778-417c-8ccd-f8852a2d79ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.026624 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 774/1558] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:11.032017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f64277-3a23-46c6-a591-04efbc9e4062 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.032017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f64277-3a23-46c6-a591-04efbc9e4062 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.032017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f64277-3a23-46c6-a591-04efbc9e4062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.039138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-7e969169-6778-417c-8ccd-f8852a2d79ec None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:11.040379 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-7e969169-6778-417c-8ccd-f8852a2d79ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:11.048195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 785/1559] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:11.048330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f9f64277-3a23-46c6-a591-04efbc9e4062 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:11.049474 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f9f64277-3a23-46c6-a591-04efbc9e4062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:11.049577 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336ad7a-5a20-43b2-9808-fa5219bd8c48 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.049757 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336ad7a-5a20-43b2-9808-fa5219bd8c48 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.050037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336ad7a-5a20-43b2-9808-fa5219bd8c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.058409 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 775/1560] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:11.059193 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f64277-3a23-46c6-a591-04efbc9e4062 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.067929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-4336ad7a-5a20-43b2-9808-fa5219bd8c48 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:11.068351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f64277-3a23-46c6-a591-04efbc9e4062 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.069116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4336ad7a-5a20-43b2-9808-fa5219bd8c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:11.072908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb3b62-4510-4104-8384-d7a938a94299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.073553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb3b62-4510-4104-8384-d7a938a94299 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.073861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb3b62-4510-4104-8384-d7a938a94299 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.074137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb3b62-4510-4104-8384-d7a938a94299 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.074854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb3b62-4510-4104-8384-d7a938a94299 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:11.075157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb3b62-4510-4104-8384-d7a938a94299 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.076761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb3b62-4510-4104-8384-d7a938a94299 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.077234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 786/1561] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:11.078019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336ad7a-5a20-43b2-9808-fa5219bd8c48 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.087613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336ad7a-5a20-43b2-9808-fa5219bd8c48 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.091968 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 776/1562] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:11.092126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354d04f-2d50-4fb9-a762-a271302d5cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.092773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354d04f-2d50-4fb9-a762-a271302d5cbb admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.093058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354d04f-2d50-4fb9-a762-a271302d5cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.093207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb3b62-4510-4104-8384-d7a938a94299 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.093317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354d04f-2d50-4fb9-a762-a271302d5cbb admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.094103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354d04f-2d50-4fb9-a762-a271302d5cbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:11.094386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354d04f-2d50-4fb9-a762-a271302d5cbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.095998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354d04f-2d50-4fb9-a762-a271302d5cbb admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.108250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 787/1563] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:11.108498 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb3b62-4510-4104-8384-d7a938a94299 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.109248 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354d04f-2d50-4fb9-a762-a271302d5cbb admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.113054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f862d8-2027-472a-92b1-7ac22976c2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.113716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f862d8-2027-472a-92b1-7ac22976c2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.114000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f862d8-2027-472a-92b1-7ac22976c2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.114279 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f862d8-2027-472a-92b1-7ac22976c2d6 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.114962 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f862d8-2027-472a-92b1-7ac22976c2d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.117652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f862d8-2027-472a-92b1-7ac22976c2d6 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.120872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354d04f-2d50-4fb9-a762-a271302d5cbb admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.125477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470d0e6-64d5-4761-a1cb-230fe77bdc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.126125 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470d0e6-64d5-4761-a1cb-230fe77bdc3d admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.126519 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470d0e6-64d5-4761-a1cb-230fe77bdc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.126722 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470d0e6-64d5-4761-a1cb-230fe77bdc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.127460 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470d0e6-64d5-4761-a1cb-230fe77bdc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.129059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470d0e6-64d5-4761-a1cb-230fe77bdc3d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.139945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 777/1564] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:11.140231 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 788/1565] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:11.141092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470d0e6-64d5-4761-a1cb-230fe77bdc3d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.144190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f862d8-2027-472a-92b1-7ac22976c2d6 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.150674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470d0e6-64d5-4761-a1cb-230fe77bdc3d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.155268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc052e77-8ca8-4aa8-ad3e-c89cfbbeb4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.155424 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f862d8-2027-472a-92b1-7ac22976c2d6 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.155990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc052e77-8ca8-4aa8-ad3e-c89cfbbeb4b2 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.156220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc052e77-8ca8-4aa8-ad3e-c89cfbbeb4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.156494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc052e77-8ca8-4aa8-ad3e-c89cfbbeb4b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.157202 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc052e77-8ca8-4aa8-ad3e-c89cfbbeb4b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:11.157508 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc052e77-8ca8-4aa8-ad3e-c89cfbbeb4b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.159070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc052e77-8ca8-4aa8-ad3e-c89cfbbeb4b2 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.159978 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef92681-adcd-446b-ba78-531b033c4dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.160720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef92681-adcd-446b-ba78-531b033c4dcf admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.161007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef92681-adcd-446b-ba78-531b033c4dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.161285 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef92681-adcd-446b-ba78-531b033c4dcf admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.162536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef92681-adcd-446b-ba78-531b033c4dcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:11.163160 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef92681-adcd-446b-ba78-531b033c4dcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.164950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef92681-adcd-446b-ba78-531b033c4dcf admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.170915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 789/1566] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:11.171714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc052e77-8ca8-4aa8-ad3e-c89cfbbeb4b2 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.185088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 778/1567] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:11.185088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef92681-adcd-446b-ba78-531b033c4dcf admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.187810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc052e77-8ca8-4aa8-ad3e-c89cfbbeb4b2 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.190065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef92681-adcd-446b-ba78-531b033c4dcf admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.192403 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb16e61-2398-4ae9-a843-fa2d6a337fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.193055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb16e61-2398-4ae9-a843-fa2d6a337fce admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.193386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb16e61-2398-4ae9-a843-fa2d6a337fce admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.193926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb16e61-2398-4ae9-a843-fa2d6a337fce admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.194646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da40d9-1320-4cb7-9526-592f751007eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.194790 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb16e61-2398-4ae9-a843-fa2d6a337fce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:11.194997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb16e61-2398-4ae9-a843-fa2d6a337fce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.195281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da40d9-1320-4cb7-9526-592f751007eb admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.195563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da40d9-1320-4cb7-9526-592f751007eb admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.195884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da40d9-1320-4cb7-9526-592f751007eb admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.196622 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da40d9-1320-4cb7-9526-592f751007eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:11.196737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb16e61-2398-4ae9-a843-fa2d6a337fce admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.196924 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da40d9-1320-4cb7-9526-592f751007eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.198475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da40d9-1320-4cb7-9526-592f751007eb admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.208402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 790/1568] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:11.209051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb16e61-2398-4ae9-a843-fa2d6a337fce admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.216691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 779/1569] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:11.216691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da40d9-1320-4cb7-9526-592f751007eb admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.220092 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da40d9-1320-4cb7-9526-592f751007eb admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.225116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41817b7-97c6-4f82-8d4e-bda79865c535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.225891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41817b7-97c6-4f82-8d4e-bda79865c535 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.226017 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb16e61-2398-4ae9-a843-fa2d6a337fce admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.226185 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41817b7-97c6-4f82-8d4e-bda79865c535 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.226457 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41817b7-97c6-4f82-8d4e-bda79865c535 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.227128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41817b7-97c6-4f82-8d4e-bda79865c535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.229025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41817b7-97c6-4f82-8d4e-bda79865c535 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.230372 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd5a27-12b2-4455-8886-bbeed9f4be29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.231008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd5a27-12b2-4455-8886-bbeed9f4be29 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.231298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd5a27-12b2-4455-8886-bbeed9f4be29 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.231558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd5a27-12b2-4455-8886-bbeed9f4be29 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.232268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd5a27-12b2-4455-8886-bbeed9f4be29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.233257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: WARNING py.warnings [None req-41cd5a27-12b2-4455-8886-bbeed9f4be29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 04:26:11.233257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: warnings.warn(msg) May 09 04:26:11.233257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: May 09 04:26:11.234435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd5a27-12b2-4455-8886-bbeed9f4be29 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.255573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 780/1570] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:11.256614 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41817b7-97c6-4f82-8d4e-bda79865c535 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.262488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 791/1571] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:11.263169 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd5a27-12b2-4455-8886-bbeed9f4be29 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.319218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41817b7-97c6-4f82-8d4e-bda79865c535 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.323694 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d984f7-fac5-4e98-ba5f-1abd25641c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.324342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd5a27-12b2-4455-8886-bbeed9f4be29 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.324573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d984f7-fac5-4e98-ba5f-1abd25641c1b admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.324857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d984f7-fac5-4e98-ba5f-1abd25641c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.325128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d984f7-fac5-4e98-ba5f-1abd25641c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.325845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d984f7-fac5-4e98-ba5f-1abd25641c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.327413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d984f7-fac5-4e98-ba5f-1abd25641c1b admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.328861 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ae688-9eb5-4f23-926b-5ff2ab1425d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.329739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae688-9eb5-4f23-926b-5ff2ab1425d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.330021 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae688-9eb5-4f23-926b-5ff2ab1425d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.330295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae688-9eb5-4f23-926b-5ff2ab1425d7 admin admin] PATH_INFO: `/v3/projects/935900d28d414052836bae2238354dcc/users/9461afbc4a9545f087184cb5e8465512/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.337202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 781/1572] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:11.337975 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d984f7-fac5-4e98-ba5f-1abd25641c1b admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.347250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ae688-9eb5-4f23-926b-5ff2ab1425d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=935900d28d414052836bae2238354dcc, user_id=9461afbc4a9545f087184cb5e8465512, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.348198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d984f7-fac5-4e98-ba5f-1abd25641c1b admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.348326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: WARNING py.warnings [None req-c41ae688-9eb5-4f23-926b-5ff2ab1425d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 04:26:11.348326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: warnings.warn(msg) May 09 04:26:11.348326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: May 09 04:26:11.351072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ae688-9eb5-4f23-926b-5ff2ab1425d7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.353242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d1234-4c0e-4475-9425-7abc4bf25bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.353861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d1234-4c0e-4475-9425-7abc4bf25bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.354140 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d1234-4c0e-4475-9425-7abc4bf25bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.354410 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d1234-4c0e-4475-9425-7abc4bf25bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.355149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d1234-4c0e-4475-9425-7abc4bf25bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.357304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d1234-4c0e-4475-9425-7abc4bf25bd6 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.363945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 792/1573] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:11 2022] PUT /identity/v3/projects/935900d28d414052836bae2238354dcc/users/9461afbc4a9545f087184cb5e8465512/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:11.364857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ae688-9eb5-4f23-926b-5ff2ab1425d7 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.384366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 782/1574] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:11.388369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d1234-4c0e-4475-9425-7abc4bf25bd6 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.427147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ae688-9eb5-4f23-926b-5ff2ab1425d7 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.447492 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d1234-4c0e-4475-9425-7abc4bf25bd6 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.452163 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0da33-84fb-4b76-adc4-5991eaf7e1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.452811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0da33-84fb-4b76-adc4-5991eaf7e1dc admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.453091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0da33-84fb-4b76-adc4-5991eaf7e1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.453364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0da33-84fb-4b76-adc4-5991eaf7e1dc admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.454099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc0da33-84fb-4b76-adc4-5991eaf7e1dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.456058 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc0da33-84fb-4b76-adc4-5991eaf7e1dc admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.481421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a88bf-1750-40c9-b608-7d55c105df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.482009 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 783/1575] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/projects => generated 369 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:11.482109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a88bf-1750-40c9-b608-7d55c105df27 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.482343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a88bf-1750-40c9-b608-7d55c105df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.482619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a88bf-1750-40c9-b608-7d55c105df27 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.482841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0da33-84fb-4b76-adc4-5991eaf7e1dc admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.483298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a88bf-1750-40c9-b608-7d55c105df27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.485226 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a88bf-1750-40c9-b608-7d55c105df27 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.514425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 793/1576] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:11.514425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a88bf-1750-40c9-b608-7d55c105df27 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.546974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0da33-84fb-4b76-adc4-5991eaf7e1dc admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.574000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a88bf-1750-40c9-b608-7d55c105df27 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.578791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3fe0d-d022-4ec9-aade-4084e250bc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.579456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3fe0d-d022-4ec9-aade-4084e250bc3f admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.579736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3fe0d-d022-4ec9-aade-4084e250bc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.580039 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3fe0d-d022-4ec9-aade-4084e250bc3f admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.580718 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3fe0d-d022-4ec9-aade-4084e250bc3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.582619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3fe0d-d022-4ec9-aade-4084e250bc3f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.603415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49e326-bce7-41ae-9b43-bbac2c5408c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.603686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 794/1577] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:11.606821 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49e326-bce7-41ae-9b43-bbac2c5408c3 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.606927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3fe0d-d022-4ec9-aade-4084e250bc3f admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.607008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49e326-bce7-41ae-9b43-bbac2c5408c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.607008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49e326-bce7-41ae-9b43-bbac2c5408c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.607008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49e326-bce7-41ae-9b43-bbac2c5408c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.607635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49e326-bce7-41ae-9b43-bbac2c5408c3 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.613827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3fe0d-d022-4ec9-aade-4084e250bc3f admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.618657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b775d5-f979-4a93-8bc4-a4be74417580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.619301 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b775d5-f979-4a93-8bc4-a4be74417580 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.619583 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b775d5-f979-4a93-8bc4-a4be74417580 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.619858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b775d5-f979-4a93-8bc4-a4be74417580 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.620559 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b775d5-f979-4a93-8bc4-a4be74417580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.622469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b775d5-f979-4a93-8bc4-a4be74417580 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.633591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 784/1578] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/projects => generated 393 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:11.634268 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49e326-bce7-41ae-9b43-bbac2c5408c3 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.643813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 795/1579] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/users => generated 287 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:11.645031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d1838-414a-4173-b96b-456138c3e060 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.645320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d1838-414a-4173-b96b-456138c3e060 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.645601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d1838-414a-4173-b96b-456138c3e060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.662674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-8f6d1838-414a-4173-b96b-456138c3e060 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:11.664107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8f6d1838-414a-4173-b96b-456138c3e060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:11.698363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49e326-bce7-41ae-9b43-bbac2c5408c3 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.702835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfb657-cd27-496b-afe3-95fe90343297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.703497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfb657-cd27-496b-afe3-95fe90343297 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.703776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfb657-cd27-496b-afe3-95fe90343297 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.704080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfb657-cd27-496b-afe3-95fe90343297 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.704790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfb657-cd27-496b-afe3-95fe90343297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.706448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfb657-cd27-496b-afe3-95fe90343297 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.716107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 785/1580] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:11.717257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfb657-cd27-496b-afe3-95fe90343297 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.726642 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfb657-cd27-496b-afe3-95fe90343297 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.727236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 796/1581] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:11.728093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d1838-414a-4173-b96b-456138c3e060 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.734016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79cd35-687a-4001-a87d-f4ea71fc6638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.734674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79cd35-687a-4001-a87d-f4ea71fc6638 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.734959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79cd35-687a-4001-a87d-f4ea71fc6638 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.735234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79cd35-687a-4001-a87d-f4ea71fc6638 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.735948 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79cd35-687a-4001-a87d-f4ea71fc6638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.737985 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79cd35-687a-4001-a87d-f4ea71fc6638 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.738110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d1838-414a-4173-b96b-456138c3e060 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.746228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a48477-620f-4747-a850-7df5c870cc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.746909 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a48477-620f-4747-a850-7df5c870cc88 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.747190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a48477-620f-4747-a850-7df5c870cc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.747466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a48477-620f-4747-a850-7df5c870cc88 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.748202 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a48477-620f-4747-a850-7df5c870cc88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.750291 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a48477-620f-4747-a850-7df5c870cc88 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.761723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 786/1582] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:11.762445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79cd35-687a-4001-a87d-f4ea71fc6638 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.773044 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79cd35-687a-4001-a87d-f4ea71fc6638 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.778586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 797/1583] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/users => generated 285 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:11.778586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a48477-620f-4747-a850-7df5c870cc88 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.778805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4863d4-6aae-4ebe-af87-5de6ead7a254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.778805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4863d4-6aae-4ebe-af87-5de6ead7a254 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.778805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4863d4-6aae-4ebe-af87-5de6ead7a254 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.778805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4863d4-6aae-4ebe-af87-5de6ead7a254 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.779405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4863d4-6aae-4ebe-af87-5de6ead7a254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.781330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4863d4-6aae-4ebe-af87-5de6ead7a254 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.785465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a48477-620f-4747-a850-7df5c870cc88 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.791617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531f716-0901-4b19-a357-21cca28d0890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.792389 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531f716-0901-4b19-a357-21cca28d0890 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.792674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531f716-0901-4b19-a357-21cca28d0890 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.792953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531f716-0901-4b19-a357-21cca28d0890 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.793859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f531f716-0901-4b19-a357-21cca28d0890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.795721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f531f716-0901-4b19-a357-21cca28d0890 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.803650 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 787/1584] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:11.804543 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4863d4-6aae-4ebe-af87-5de6ead7a254 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.810250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 798/1585] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:11.810250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531f716-0901-4b19-a357-21cca28d0890 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.814264 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4863d4-6aae-4ebe-af87-5de6ead7a254 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.818353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531f716-0901-4b19-a357-21cca28d0890 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.819160 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b432b5-99d7-49c6-98e3-038f0a177ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.819821 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b432b5-99d7-49c6-98e3-038f0a177ccd admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.820129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b432b5-99d7-49c6-98e3-038f0a177ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.820400 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b432b5-99d7-49c6-98e3-038f0a177ccd admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.821084 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b432b5-99d7-49c6-98e3-038f0a177ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.823031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692e2c1-885c-43ec-b1bd-f88f6fb98244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.823168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b432b5-99d7-49c6-98e3-038f0a177ccd admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.823708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692e2c1-885c-43ec-b1bd-f88f6fb98244 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.824022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692e2c1-885c-43ec-b1bd-f88f6fb98244 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.824299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692e2c1-885c-43ec-b1bd-f88f6fb98244 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.825067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692e2c1-885c-43ec-b1bd-f88f6fb98244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.826635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692e2c1-885c-43ec-b1bd-f88f6fb98244 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.839035 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 799/1586] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:11.840069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692e2c1-885c-43ec-b1bd-f88f6fb98244 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.844194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 788/1587] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:11 2022] POST /identity/v3/users => generated 297 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:11.845024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b432b5-99d7-49c6-98e3-038f0a177ccd admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.846574 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-18b432b5-99d7-49c6-98e3-038f0a177ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:11.850658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692e2c1-885c-43ec-b1bd-f88f6fb98244 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.855228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e63bb3-61f0-4fa1-bcd9-1e6538e400f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.856144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e63bb3-61f0-4fa1-bcd9-1e6538e400f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.856418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e63bb3-61f0-4fa1-bcd9-1e6538e400f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.856723 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e63bb3-61f0-4fa1-bcd9-1e6538e400f6 admin admin] PATH_INFO: `/v3/projects/3c23c235fa1c4944bd7a337357d0c030/users/85429c28c994443eb30b98ece56c4c9b/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.871920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e63bb3-61f0-4fa1-bcd9-1e6538e400f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c23c235fa1c4944bd7a337357d0c030, user_id=85429c28c994443eb30b98ece56c4c9b, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.874553 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e63bb3-61f0-4fa1-bcd9-1e6538e400f6 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.887000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 800/1588] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:11 2022] PUT /identity/v3/projects/3c23c235fa1c4944bd7a337357d0c030/users/85429c28c994443eb30b98ece56c4c9b/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:11.887818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e63bb3-61f0-4fa1-bcd9-1e6538e400f6 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.904780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b432b5-99d7-49c6-98e3-038f0a177ccd admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.946508 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e63bb3-61f0-4fa1-bcd9-1e6538e400f6 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.951220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c876d7-7963-4b6d-be46-5437252bd6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.951377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c252c7-5405-4a2f-a8fa-9037dfd9c14c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.951973 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c252c7-5405-4a2f-a8fa-9037dfd9c14c service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.952197 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c876d7-7963-4b6d-be46-5437252bd6af admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.952315 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c252c7-5405-4a2f-a8fa-9037dfd9c14c service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.952515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c252c7-5405-4a2f-a8fa-9037dfd9c14c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.952750 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c876d7-7963-4b6d-be46-5437252bd6af admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.953117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c876d7-7963-4b6d-be46-5437252bd6af admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.953934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c876d7-7963-4b6d-be46-5437252bd6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.955624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c876d7-7963-4b6d-be46-5437252bd6af admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.961662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c252c7-5405-4a2f-a8fa-9037dfd9c14c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.963063 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c252c7-5405-4a2f-a8fa-9037dfd9c14c service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.965622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 801/1589] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:11.966409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c876d7-7963-4b6d-be46-5437252bd6af admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.976634 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 789/1590] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:11.977518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c252c7-5405-4a2f-a8fa-9037dfd9c14c service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:11.978267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c876d7-7963-4b6d-be46-5437252bd6af admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.984143 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfba507-da77-44e9-bf15-b5b1fdc17824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.984937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfba507-da77-44e9-bf15-b5b1fdc17824 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.985315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfba507-da77-44e9-bf15-b5b1fdc17824 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.985688 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfba507-da77-44e9-bf15-b5b1fdc17824 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.986524 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfba507-da77-44e9-bf15-b5b1fdc17824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.986752 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c252c7-5405-4a2f-a8fa-9037dfd9c14c service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:11.988330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfba507-da77-44e9-bf15-b5b1fdc17824 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:11.994021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d20a12-1d6a-4c15-80d5-71851f9b6f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:11.994688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d20a12-1d6a-4c15-80d5-71851f9b6f90 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:11.994975 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d20a12-1d6a-4c15-80d5-71851f9b6f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:11.995248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d20a12-1d6a-4c15-80d5-71851f9b6f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:11.996029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d20a12-1d6a-4c15-80d5-71851f9b6f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:11.997676 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d20a12-1d6a-4c15-80d5-71851f9b6f90 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.003775 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 802/1591] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:12.004507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfba507-da77-44e9-bf15-b5b1fdc17824 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.009156 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 790/1592] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:12.012830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d20a12-1d6a-4c15-80d5-71851f9b6f90 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.015462 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfba507-da77-44e9-bf15-b5b1fdc17824 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.020482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0ba2d-91e3-4d8c-85db-8717d9105934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.021168 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0ba2d-91e3-4d8c-85db-8717d9105934 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.022025 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0ba2d-91e3-4d8c-85db-8717d9105934 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.022310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0ba2d-91e3-4d8c-85db-8717d9105934 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.023031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0ba2d-91e3-4d8c-85db-8717d9105934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.023193 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d20a12-1d6a-4c15-80d5-71851f9b6f90 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.024667 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0ba2d-91e3-4d8c-85db-8717d9105934 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.028306 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21678bc-9c2a-4f6b-8d6f-23a3b930dff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.029337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21678bc-9c2a-4f6b-8d6f-23a3b930dff9 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.029337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21678bc-9c2a-4f6b-8d6f-23a3b930dff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.029555 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21678bc-9c2a-4f6b-8d6f-23a3b930dff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.030223 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21678bc-9c2a-4f6b-8d6f-23a3b930dff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.031794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21678bc-9c2a-4f6b-8d6f-23a3b930dff9 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.035505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 803/1593] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:12.036620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0ba2d-91e3-4d8c-85db-8717d9105934 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.045251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 791/1594] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:12.045251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09a5b6-671b-484b-9e87-9c9fd76d5b95 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.045251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09a5b6-671b-484b-9e87-9c9fd76d5b95 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.045251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09a5b6-671b-484b-9e87-9c9fd76d5b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.050067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0ba2d-91e3-4d8c-85db-8717d9105934 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.056516 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126d1a2-df49-42ff-b31e-e3cef2c34801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.057295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126d1a2-df49-42ff-b31e-e3cef2c34801 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.057666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126d1a2-df49-42ff-b31e-e3cef2c34801 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.058002 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126d1a2-df49-42ff-b31e-e3cef2c34801 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.058901 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126d1a2-df49-42ff-b31e-e3cef2c34801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.060619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126d1a2-df49-42ff-b31e-e3cef2c34801 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.062751 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-6c09a5b6-671b-484b-9e87-9c9fd76d5b95 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:12.064112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-6c09a5b6-671b-484b-9e87-9c9fd76d5b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:12.075875 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 804/1595] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:12.076096 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 792/1596] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:12.076219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126d1a2-df49-42ff-b31e-e3cef2c34801 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.076322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09a5b6-671b-484b-9e87-9c9fd76d5b95 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.085481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09a5b6-671b-484b-9e87-9c9fd76d5b95 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.091021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b96c31-7fb0-4762-808d-f43c3b53f93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.091737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b96c31-7fb0-4762-808d-f43c3b53f93c admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.092068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b96c31-7fb0-4762-808d-f43c3b53f93c admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.092347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b96c31-7fb0-4762-808d-f43c3b53f93c admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.092626 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126d1a2-df49-42ff-b31e-e3cef2c34801 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.093079 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b96c31-7fb0-4762-808d-f43c3b53f93c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.094726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b96c31-7fb0-4762-808d-f43c3b53f93c admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.097570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96057e8e-d404-4b7b-9a05-a12161682669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.098236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96057e8e-d404-4b7b-9a05-a12161682669 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.098517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96057e8e-d404-4b7b-9a05-a12161682669 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.098791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96057e8e-d404-4b7b-9a05-a12161682669 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.099686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96057e8e-d404-4b7b-9a05-a12161682669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.101342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96057e8e-d404-4b7b-9a05-a12161682669 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.105170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 793/1597] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:12.105927 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b96c31-7fb0-4762-808d-f43c3b53f93c admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.117046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b96c31-7fb0-4762-808d-f43c3b53f93c admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.120437 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 805/1598] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:12.121418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96057e8e-d404-4b7b-9a05-a12161682669 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.122501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb6c1d-7eec-4f4e-814e-4ced432d8fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.123285 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb6c1d-7eec-4f4e-814e-4ced432d8fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.123616 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb6c1d-7eec-4f4e-814e-4ced432d8fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.123994 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb6c1d-7eec-4f4e-814e-4ced432d8fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.124793 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb6c1d-7eec-4f4e-814e-4ced432d8fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.126472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb6c1d-7eec-4f4e-814e-4ced432d8fa3 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.133375 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96057e8e-d404-4b7b-9a05-a12161682669 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.137683 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99de32-2a09-4915-b91d-a30d13ed4ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.138518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 794/1599] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:12.138998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99de32-2a09-4915-b91d-a30d13ed4ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.139382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99de32-2a09-4915-b91d-a30d13ed4ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.139660 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99de32-2a09-4915-b91d-a30d13ed4ce8 admin admin] PATH_INFO: `/v3/projects/5e56ba7f9cc64897a1d95ac15061e9e4/users/4f8f6c77f898475e8acf1c4042c3928b/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.141174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb6c1d-7eec-4f4e-814e-4ced432d8fa3 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.153168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb6c1d-7eec-4f4e-814e-4ced432d8fa3 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.156890 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99de32-2a09-4915-b91d-a30d13ed4ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e56ba7f9cc64897a1d95ac15061e9e4, user_id=4f8f6c77f898475e8acf1c4042c3928b, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.158812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3e5cb-5a0d-411b-a2e4-130863a813dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.159803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3e5cb-5a0d-411b-a2e4-130863a813dc admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.160243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3e5cb-5a0d-411b-a2e4-130863a813dc admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.160491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99de32-2a09-4915-b91d-a30d13ed4ce8 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.160608 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3e5cb-5a0d-411b-a2e4-130863a813dc admin admin] PATH_INFO: `/v3/projects/d8b020b16f9c43cb9e04a786011069c4/users/8381a286bb624131b5a610de86488e84/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.182140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 806/1600] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:12 2022] PUT /identity/v3/projects/5e56ba7f9cc64897a1d95ac15061e9e4/users/4f8f6c77f898475e8acf1c4042c3928b/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:12.182140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99de32-2a09-4915-b91d-a30d13ed4ce8 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.182502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3e5cb-5a0d-411b-a2e4-130863a813dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b020b16f9c43cb9e04a786011069c4, user_id=8381a286bb624131b5a610de86488e84, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.186530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3e5cb-5a0d-411b-a2e4-130863a813dc admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.202791 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 795/1601] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:12 2022] PUT /identity/v3/projects/d8b020b16f9c43cb9e04a786011069c4/users/8381a286bb624131b5a610de86488e84/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:12.203387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3e5cb-5a0d-411b-a2e4-130863a813dc admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.255511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99de32-2a09-4915-b91d-a30d13ed4ce8 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.265240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3e5cb-5a0d-411b-a2e4-130863a813dc admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.269451 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6b193-614c-4a48-bfe8-6984d4384185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.270204 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6b193-614c-4a48-bfe8-6984d4384185 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.270514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6b193-614c-4a48-bfe8-6984d4384185 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.270790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6b193-614c-4a48-bfe8-6984d4384185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.317846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25f37e-f358-4169-9dcd-c07345fc1f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.319138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25f37e-f358-4169-9dcd-c07345fc1f0a admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.319473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25f37e-f358-4169-9dcd-c07345fc1f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.319752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25f37e-f358-4169-9dcd-c07345fc1f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.320512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25f37e-f358-4169-9dcd-c07345fc1f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.322123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25f37e-f358-4169-9dcd-c07345fc1f0a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.335884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 807/1602] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:12.335884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25f37e-f358-4169-9dcd-c07345fc1f0a admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.342016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6b193-614c-4a48-bfe8-6984d4384185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.343613 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6b193-614c-4a48-bfe8-6984d4384185 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.351081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25f37e-f358-4169-9dcd-c07345fc1f0a admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.355057 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 796/1603] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:12.355791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecb431-7e4b-4bcb-990e-5175368431da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.355966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6b193-614c-4a48-bfe8-6984d4384185 service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.356844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecb431-7e4b-4bcb-990e-5175368431da admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.357245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecb431-7e4b-4bcb-990e-5175368431da admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.357554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecb431-7e4b-4bcb-990e-5175368431da admin admin] PATH_INFO: `/v3/projects/ff4a73109fde49f994f89b9df5041a34/users/63a787dd74a34eedacf2aea3de2f5c90/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.366316 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6b193-614c-4a48-bfe8-6984d4384185 service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.372561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745234b-005d-4417-80f5-b7cda9f827fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.373590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745234b-005d-4417-80f5-b7cda9f827fd admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.373971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745234b-005d-4417-80f5-b7cda9f827fd admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.374348 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745234b-005d-4417-80f5-b7cda9f827fd admin admin] PATH_INFO: `/v3/projects/4b289587284a4aba96b85ffec328efff/users/3fd3e173caa849b98a2e58339001e0b7/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.376037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecb431-7e4b-4bcb-990e-5175368431da admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4a73109fde49f994f89b9df5041a34, user_id=63a787dd74a34eedacf2aea3de2f5c90, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.379433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecb431-7e4b-4bcb-990e-5175368431da admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.392255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745234b-005d-4417-80f5-b7cda9f827fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b289587284a4aba96b85ffec328efff, user_id=3fd3e173caa849b98a2e58339001e0b7, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.394989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745234b-005d-4417-80f5-b7cda9f827fd admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.419960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 808/1604] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:12 2022] PUT /identity/v3/projects/ff4a73109fde49f994f89b9df5041a34/users/63a787dd74a34eedacf2aea3de2f5c90/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:12.420478 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 797/1605] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:12 2022] PUT /identity/v3/projects/4b289587284a4aba96b85ffec328efff/users/3fd3e173caa849b98a2e58339001e0b7/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:12.420776 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecb431-7e4b-4bcb-990e-5175368431da admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.420880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf0d7-d51d-475d-9fcd-199503717f75 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.420880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf0d7-d51d-475d-9fcd-199503717f75 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.420880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf0d7-d51d-475d-9fcd-199503717f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.435774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-5bbaf0d7-d51d-475d-9fcd-199503717f75 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:12.437343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5bbaf0d7-d51d-475d-9fcd-199503717f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:12.512561 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecb431-7e4b-4bcb-990e-5175368431da admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.548812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 798/1606] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:12.551201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bd366-caaf-4f80-8c2b-d0b2709577ae None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.551909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bd366-caaf-4f80-8c2b-d0b2709577ae None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.554586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bd366-caaf-4f80-8c2b-d0b2709577ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.595155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f82bd366-caaf-4f80-8c2b-d0b2709577ae None None] MFA Rules not processed for user `8381a286bb624131b5a610de86488e84`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:12.595369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e6b98-a1b5-4cf3-9d4e-5e1e5e02675a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.596369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e6b98-a1b5-4cf3-9d4e-5e1e5e02675a admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.596502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f82bd366-caaf-4f80-8c2b-d0b2709577ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:12.596813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e6b98-a1b5-4cf3-9d4e-5e1e5e02675a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.597265 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e6b98-a1b5-4cf3-9d4e-5e1e5e02675a admin admin] PATH_INFO: `/v3/projects/aee7fe01f201408e8949d3405fe44586/users/c28066d9a3ab4d58aa9e950abb9ec2c9/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.613307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e6b98-a1b5-4cf3-9d4e-5e1e5e02675a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee7fe01f201408e8949d3405fe44586, user_id=c28066d9a3ab4d58aa9e950abb9ec2c9, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.617470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e6b98-a1b5-4cf3-9d4e-5e1e5e02675a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.639032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 809/1607] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:12 2022] PUT /identity/v3/projects/aee7fe01f201408e8949d3405fe44586/users/c28066d9a3ab4d58aa9e950abb9ec2c9/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:12.639956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e6b98-a1b5-4cf3-9d4e-5e1e5e02675a admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.669698 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 799/1608] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:12.671568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85269e-3d09-4771-af2c-9a4a4f00e88c None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.672365 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85269e-3d09-4771-af2c-9a4a4f00e88c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.672506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85269e-3d09-4771-af2c-9a4a4f00e88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.702647 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-cd85269e-3d09-4771-af2c-9a4a4f00e88c None None] MFA Rules not processed for user `63a787dd74a34eedacf2aea3de2f5c90`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:12.704111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-cd85269e-3d09-4771-af2c-9a4a4f00e88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:12.781341 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e6b98-a1b5-4cf3-9d4e-5e1e5e02675a admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.790803 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642f719-ed24-46b2-8907-1b108757daf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.792956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642f719-ed24-46b2-8907-1b108757daf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.794515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642f719-ed24-46b2-8907-1b108757daf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.795402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642f719-ed24-46b2-8907-1b108757daf7 admin admin] PATH_INFO: `/v3/projects/d28d5139952d450fa3b7da38a62d7a03/users/87b7c3ec77d4493dac92bb61d0b376be/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.809881 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 800/1609] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:12.813504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0828f0d-3239-4da9-b3f2-e1771e3f2d5e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.813870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0828f0d-3239-4da9-b3f2-e1771e3f2d5e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.814081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0828f0d-3239-4da9-b3f2-e1771e3f2d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.836792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9642f719-ed24-46b2-8907-1b108757daf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d28d5139952d450fa3b7da38a62d7a03, user_id=87b7c3ec77d4493dac92bb61d0b376be, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:12.840217 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9642f719-ed24-46b2-8907-1b108757daf7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:12.859151 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 810/1610] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:12 2022] PUT /identity/v3/projects/d28d5139952d450fa3b7da38a62d7a03/users/87b7c3ec77d4493dac92bb61d0b376be/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:12.860193 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642f719-ed24-46b2-8907-1b108757daf7 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:12.872129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-e0828f0d-3239-4da9-b3f2-e1771e3f2d5e None None] MFA Rules not processed for user `3fd3e173caa849b98a2e58339001e0b7`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:12.873735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e0828f0d-3239-4da9-b3f2-e1771e3f2d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:12.926641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642f719-ed24-46b2-8907-1b108757daf7 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:12.931931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aff559-3eca-4756-b77d-b9bc8cb18dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:12.932678 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aff559-3eca-4756-b77d-b9bc8cb18dad service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.933061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aff559-3eca-4756-b77d-b9bc8cb18dad service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.933423 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aff559-3eca-4756-b77d-b9bc8cb18dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:12.982859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 801/1611] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:12.984228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f716a-5fdf-4790-bc83-976c88b5efa3 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:12.984568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f716a-5fdf-4790-bc83-976c88b5efa3 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:12.984852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f716a-5fdf-4790-bc83-976c88b5efa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.011157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aff559-3eca-4756-b77d-b9bc8cb18dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:13.012929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aff559-3eca-4756-b77d-b9bc8cb18dad service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:13.026824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 811/1612] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:13.034472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-a93f716a-5fdf-4790-bc83-976c88b5efa3 None None] MFA Rules not processed for user `c28066d9a3ab4d58aa9e950abb9ec2c9`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:13.036120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a93f716a-5fdf-4790-bc83-976c88b5efa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:13.040979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0295e-bc53-4ec2-8401-db1dc1e951bf None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.043444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0295e-bc53-4ec2-8401-db1dc1e951bf None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.044251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0295e-bc53-4ec2-8401-db1dc1e951bf None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.050114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 812/1613] 10.222.0.44 () {54 vars in 961 bytes} [Mon May 9 04:26:13 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:26:13.051146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0295e-bc53-4ec2-8401-db1dc1e951bf None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:13.083271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0295e-bc53-4ec2-8401-db1dc1e951bf None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:13.089347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806dcd2c-b708-481b-9a61-4bbe09df7ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:13.090086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dcd2c-b708-481b-9a61-4bbe09df7ada admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.091012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dcd2c-b708-481b-9a61-4bbe09df7ada admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.091298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806dcd2c-b708-481b-9a61-4bbe09df7ada admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.092034 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806dcd2c-b708-481b-9a61-4bbe09df7ada admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:13.094923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806dcd2c-b708-481b-9a61-4bbe09df7ada admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:13.134601 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 802/1614] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:13.138796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef61fd-e1c1-4112-88cc-4371a76b80f8 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.139182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 813/1615] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:13 2022] POST /identity/v3/projects => generated 367 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:13.139502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef61fd-e1c1-4112-88cc-4371a76b80f8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.140348 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef61fd-e1c1-4112-88cc-4371a76b80f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.143497 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7b8a3-f389-4a0d-bdc9-8b36050e596c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.143872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7b8a3-f389-4a0d-bdc9-8b36050e596c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.144595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7b8a3-f389-4a0d-bdc9-8b36050e596c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.169111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-63c7b8a3-f389-4a0d-bdc9-8b36050e596c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:13.170497 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-63c7b8a3-f389-4a0d-bdc9-8b36050e596c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:13.179730 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-05ef61fd-e1c1-4112-88cc-4371a76b80f8 None None] MFA Rules not processed for user `9461afbc4a9545f087184cb5e8465512`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:13.181362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-05ef61fd-e1c1-4112-88cc-4371a76b80f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:13.264560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 814/1616] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:13.264560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe4cb6-1458-468c-858f-a8ef5c37262c None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.264881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe4cb6-1458-468c-858f-a8ef5c37262c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.265310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe4cb6-1458-468c-858f-a8ef5c37262c None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.267256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 815/1617] 10.222.0.44 () {54 vars in 961 bytes} [Mon May 9 04:26:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:26:13.269104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ecb7c-8375-4413-8cff-f233af5d3093 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.271042 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ecb7c-8375-4413-8cff-f233af5d3093 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.272568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ecb7c-8375-4413-8cff-f233af5d3093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.283081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 803/1618] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:13.283645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef61fd-e1c1-4112-88cc-4371a76b80f8 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:13.323317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef61fd-e1c1-4112-88cc-4371a76b80f8 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:13.335921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-367ecb7c-8375-4413-8cff-f233af5d3093 None None] MFA Rules not processed for user `faf6107735424ff7901614953e7df403`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:13.336822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02128a5-6392-4378-aaa1-dfc665297e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:13.337511 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02128a5-6392-4378-aaa1-dfc665297e50 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.337650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-367ecb7c-8375-4413-8cff-f233af5d3093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:13.337843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02128a5-6392-4378-aaa1-dfc665297e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.338113 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02128a5-6392-4378-aaa1-dfc665297e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.338742 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02128a5-6392-4378-aaa1-dfc665297e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:13.345691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02128a5-6392-4378-aaa1-dfc665297e50 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:13.394428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 804/1619] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:13 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:13.395302 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02128a5-6392-4378-aaa1-dfc665297e50 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:13.431944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 816/1620] 10.222.0.44 () {58 vars in 1058 bytes} [Mon May 9 04:26:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:13.433269 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ecb7c-8375-4413-8cff-f233af5d3093 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:13.485414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02128a5-6392-4378-aaa1-dfc665297e50 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:13.491638 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407a60c-15ea-488d-81bc-961acca3d261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:13.492479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407a60c-15ea-488d-81bc-961acca3d261 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.493054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407a60c-15ea-488d-81bc-961acca3d261 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.493387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407a60c-15ea-488d-81bc-961acca3d261 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.494181 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407a60c-15ea-488d-81bc-961acca3d261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:13.496339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407a60c-15ea-488d-81bc-961acca3d261 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:13.508123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ecb7c-8375-4413-8cff-f233af5d3093 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:13.529281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 805/1621] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:13 2022] POST /identity/v3/users => generated 290 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:13.530611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11de6bd-2db4-4a8b-9994-f3473cab359e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.530871 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11de6bd-2db4-4a8b-9994-f3473cab359e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.531156 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11de6bd-2db4-4a8b-9994-f3473cab359e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.555133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d11de6bd-2db4-4a8b-9994-f3473cab359e None None] MFA Rules not processed for user `faf6107735424ff7901614953e7df403`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:13.556835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d11de6bd-2db4-4a8b-9994-f3473cab359e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:13.580198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8090665-411f-40f5-9bc4-d8f5118081fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:13.581580 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8090665-411f-40f5-9bc4-d8f5118081fe service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.582011 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8090665-411f-40f5-9bc4-d8f5118081fe service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.582814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8090665-411f-40f5-9bc4-d8f5118081fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.594109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8090665-411f-40f5-9bc4-d8f5118081fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:13.595884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8090665-411f-40f5-9bc4-d8f5118081fe service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:13.608275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 817/1622] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:13.610095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f34afa-4545-4734-a065-0152df181e5d None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.610558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f34afa-4545-4734-a065-0152df181e5d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.610971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f34afa-4545-4734-a065-0152df181e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.629920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-30f34afa-4545-4734-a065-0152df181e5d None None] MFA Rules not processed for user `9461afbc4a9545f087184cb5e8465512`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:13.631523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-30f34afa-4545-4734-a065-0152df181e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:13.642193 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 806/1623] 10.222.0.44 () {58 vars in 1058 bytes} [Mon May 9 04:26:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:13.642903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11de6bd-2db4-4a8b-9994-f3473cab359e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:13.657282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11de6bd-2db4-4a8b-9994-f3473cab359e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:13.664976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b12f5-8cc4-4e8f-ae7e-b1d7230f11dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:13.666285 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b12f5-8cc4-4e8f-ae7e-b1d7230f11dd admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.666543 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b12f5-8cc4-4e8f-ae7e-b1d7230f11dd admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.666825 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b12f5-8cc4-4e8f-ae7e-b1d7230f11dd admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.667768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b12f5-8cc4-4e8f-ae7e-b1d7230f11dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:13.671806 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b12f5-8cc4-4e8f-ae7e-b1d7230f11dd admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:13.702786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 807/1624] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:13 2022] POST /identity/v3/users => generated 293 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:13.708144 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b12f5-8cc4-4e8f-ae7e-b1d7230f11dd admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:13.729675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b12f5-8cc4-4e8f-ae7e-b1d7230f11dd admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:13.740358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cb107-6b5c-4df4-9f62-b0103a7b8081 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:13.741857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cb107-6b5c-4df4-9f62-b0103a7b8081 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.742195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cb107-6b5c-4df4-9f62-b0103a7b8081 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.742476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cb107-6b5c-4df4-9f62-b0103a7b8081 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.749758 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 818/1625] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:13.758620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f34afa-4545-4734-a065-0152df181e5d None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:13.772012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f34afa-4545-4734-a065-0152df181e5d None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:13.779320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf4961-e776-4850-9601-11498d09af44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:13.780047 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf4961-e776-4850-9601-11498d09af44 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.780335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf4961-e776-4850-9601-11498d09af44 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.780710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf4961-e776-4850-9601-11498d09af44 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.781467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf4961-e776-4850-9601-11498d09af44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:13.783359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf4961-e776-4850-9601-11498d09af44 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:13.823291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cb107-6b5c-4df4-9f62-b0103a7b8081 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:13.826602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cb107-6b5c-4df4-9f62-b0103a7b8081 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:13.830129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 819/1626] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:13.831125 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf4961-e776-4850-9601-11498d09af44 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:13.856388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 808/1627] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:13.857299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cb107-6b5c-4df4-9f62-b0103a7b8081 service nova] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:13.858327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf4961-e776-4850-9601-11498d09af44 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:13.868223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c828e4-b088-40b5-805a-143edd75eab9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:13.868903 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c828e4-b088-40b5-805a-143edd75eab9 service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.869192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c828e4-b088-40b5-805a-143edd75eab9 service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.869474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c828e4-b088-40b5-805a-143edd75eab9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.873867 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cb107-6b5c-4df4-9f62-b0103a7b8081 service nova] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:13.888128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8b8b-4c88-4337-9eb4-6b84311395e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:13.888982 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8b8b-4c88-4337-9eb4-6b84311395e4 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.889228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8b8b-4c88-4337-9eb4-6b84311395e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.889416 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8b8b-4c88-4337-9eb4-6b84311395e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.890068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf8b8b-4c88-4337-9eb4-6b84311395e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:13.891670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf8b8b-4c88-4337-9eb4-6b84311395e4 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:13.910379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 809/1628] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:13.910379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8b8b-4c88-4337-9eb4-6b84311395e4 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:13.947270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8b8b-4c88-4337-9eb4-6b84311395e4 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:13.952558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dd7a6-9049-4a90-95f9-c4b30c0645b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:13.953609 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dd7a6-9049-4a90-95f9-c4b30c0645b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:13.953894 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dd7a6-9049-4a90-95f9-c4b30c0645b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:13.954169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dd7a6-9049-4a90-95f9-c4b30c0645b4 admin admin] PATH_INFO: `/v3/projects/8519390d3262470593b9acf3ab8f240a/users/19ee2e5b9b984391be2d622c2718cc4a/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:13.984341 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dd7a6-9049-4a90-95f9-c4b30c0645b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8519390d3262470593b9acf3ab8f240a, user_id=19ee2e5b9b984391be2d622c2718cc4a, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:13.987488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dd7a6-9049-4a90-95f9-c4b30c0645b4 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:14.008610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c828e4-b088-40b5-805a-143edd75eab9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:14.013524 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c828e4-b088-40b5-805a-143edd75eab9 service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:14.031157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 810/1629] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:13 2022] PUT /identity/v3/projects/8519390d3262470593b9acf3ab8f240a/users/19ee2e5b9b984391be2d622c2718cc4a/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:14.032032 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dd7a6-9049-4a90-95f9-c4b30c0645b4 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:14.041215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 820/1630] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:14.041215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c828e4-b088-40b5-805a-143edd75eab9 service nova] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:14.065399 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c828e4-b088-40b5-805a-143edd75eab9 service nova] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:14.194231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dd7a6-9049-4a90-95f9-c4b30c0645b4 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:14.197976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189376c-9781-4014-97a4-ad40bd370db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:14.199087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189376c-9781-4014-97a4-ad40bd370db8 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:14.199540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189376c-9781-4014-97a4-ad40bd370db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:14.199912 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189376c-9781-4014-97a4-ad40bd370db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:14.200710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189376c-9781-4014-97a4-ad40bd370db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:14.201673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75804b4-77e9-40d1-8ab3-d935575618d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:14.202566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189376c-9781-4014-97a4-ad40bd370db8 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:14.203003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75804b4-77e9-40d1-8ab3-d935575618d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:14.203354 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75804b4-77e9-40d1-8ab3-d935575618d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:14.203705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75804b4-77e9-40d1-8ab3-d935575618d3 admin admin] PATH_INFO: `/v3/projects/8c10981ed3e443fd99a7241783c95882/users/597169257ff94108832959dc37f27164/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:14.225419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 821/1631] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:14.230022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189376c-9781-4014-97a4-ad40bd370db8 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:14.230565 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75804b4-77e9-40d1-8ab3-d935575618d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c10981ed3e443fd99a7241783c95882, user_id=597169257ff94108832959dc37f27164, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:14.235273 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75804b4-77e9-40d1-8ab3-d935575618d3 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:14.248528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189376c-9781-4014-97a4-ad40bd370db8 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:14.260936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 811/1632] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:14 2022] PUT /identity/v3/projects/8c10981ed3e443fd99a7241783c95882/users/597169257ff94108832959dc37f27164/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:14.266168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75804b4-77e9-40d1-8ab3-d935575618d3 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:14.375507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6986f-9cee-4a9a-80f7-35bc920ed7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:14.376761 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6986f-9cee-4a9a-80f7-35bc920ed7dd admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:14.377149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6986f-9cee-4a9a-80f7-35bc920ed7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:14.377574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6986f-9cee-4a9a-80f7-35bc920ed7dd admin admin] PATH_INFO: `/v3/projects/8519390d3262470593b9acf3ab8f240a/users/19ee2e5b9b984391be2d622c2718cc4a/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:14.384248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75804b4-77e9-40d1-8ab3-d935575618d3 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:14.384559 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6986f-9cee-4a9a-80f7-35bc920ed7dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8519390d3262470593b9acf3ab8f240a, user_id=19ee2e5b9b984391be2d622c2718cc4a, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:14.391678 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6986f-9cee-4a9a-80f7-35bc920ed7dd admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:14.392347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad375eef-7851-4b29-aa85-180cf04759fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:14.393049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad375eef-7851-4b29-aa85-180cf04759fe admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:14.393337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad375eef-7851-4b29-aa85-180cf04759fe admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:14.393621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad375eef-7851-4b29-aa85-180cf04759fe admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:14.395557 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad375eef-7851-4b29-aa85-180cf04759fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:14.399650 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad375eef-7851-4b29-aa85-180cf04759fe admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:14.451211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 822/1633] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:14 2022] PUT /identity/v3/projects/8519390d3262470593b9acf3ab8f240a/users/19ee2e5b9b984391be2d622c2718cc4a/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:14.451587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 812/1634] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:14.451690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6986f-9cee-4a9a-80f7-35bc920ed7dd admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:14.456119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a449309-0870-4db4-9edc-d467bed6f035 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:14.456468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a449309-0870-4db4-9edc-d467bed6f035 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:14.456804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a449309-0870-4db4-9edc-d467bed6f035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:14.504849 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-9a449309-0870-4db4-9edc-d467bed6f035 None None] MFA Rules not processed for user `19ee2e5b9b984391be2d622c2718cc4a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:14.507715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9a449309-0870-4db4-9edc-d467bed6f035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:14.637010 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 813/1635] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:14.638488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374fe0e-4c1a-411e-a351-611af9a2d719 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:14.638840 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374fe0e-4c1a-411e-a351-611af9a2d719 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:14.639183 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374fe0e-4c1a-411e-a351-611af9a2d719 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:14.640981 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 814/1636] 10.222.0.44 () {54 vars in 912 bytes} [Mon May 9 04:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:26:14.653928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e1850-19c9-4dee-ae9d-a604c21f193c None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:14.655112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e1850-19c9-4dee-ae9d-a604c21f193c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:14.656414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e1850-19c9-4dee-ae9d-a604c21f193c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:14.662237 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6986f-9cee-4a9a-80f7-35bc920ed7dd admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:14.693784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3908348-7d19-4f31-9694-b398a010f8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:14.702310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3908348-7d19-4f31-9694-b398a010f8bb admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:14.702687 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3908348-7d19-4f31-9694-b398a010f8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:14.703886 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3908348-7d19-4f31-9694-b398a010f8bb admin admin] PATH_INFO: `/v3/projects/8c10981ed3e443fd99a7241783c95882/users/597169257ff94108832959dc37f27164/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:14.706303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-be0e1850-19c9-4dee-ae9d-a604c21f193c None None] MFA Rules not processed for user `faf6107735424ff7901614953e7df403`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:14.707544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-be0e1850-19c9-4dee-ae9d-a604c21f193c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:14.712928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3908348-7d19-4f31-9694-b398a010f8bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c10981ed3e443fd99a7241783c95882, user_id=597169257ff94108832959dc37f27164, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:14.730957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3908348-7d19-4f31-9694-b398a010f8bb admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:14.754297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 823/1637] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:14 2022] PUT /identity/v3/projects/8c10981ed3e443fd99a7241783c95882/users/597169257ff94108832959dc37f27164/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:14.761014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3908348-7d19-4f31-9694-b398a010f8bb admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:14.892956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3908348-7d19-4f31-9694-b398a010f8bb admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:14.903840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111525c-2092-4f6c-a074-ab76d1965a40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:14.904551 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111525c-2092-4f6c-a074-ab76d1965a40 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:14.904836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111525c-2092-4f6c-a074-ab76d1965a40 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:14.905113 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111525c-2092-4f6c-a074-ab76d1965a40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:14.940943 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 815/1638] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:26:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:14.942965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0af64-1248-4b15-a38c-d02445eaff85 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:14.944852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0af64-1248-4b15-a38c-d02445eaff85 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:14.945480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0af64-1248-4b15-a38c-d02445eaff85 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:14.951268 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 816/1639] 10.222.0.44 () {54 vars in 912 bytes} [Mon May 9 04:26:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:26:14.953909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e94a79-7054-43a4-9e61-6381a0143ea3 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:14.958454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e94a79-7054-43a4-9e61-6381a0143ea3 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:14.958901 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e94a79-7054-43a4-9e61-6381a0143ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:15.015534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-c4e94a79-7054-43a4-9e61-6381a0143ea3 None None] MFA Rules not processed for user `597169257ff94108832959dc37f27164`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:15.016935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c4e94a79-7054-43a4-9e61-6381a0143ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:15.035685 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111525c-2092-4f6c-a074-ab76d1965a40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:15.043568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111525c-2092-4f6c-a074-ab76d1965a40 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:15.089476 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 824/1640] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:15.090842 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d8991-3c9f-4690-b62b-3616eb976d71 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:15.091130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d8991-3c9f-4690-b62b-3616eb976d71 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:15.091915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d8991-3c9f-4690-b62b-3616eb976d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:15.114478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-125d8991-3c9f-4690-b62b-3616eb976d71 None None] MFA Rules not processed for user `faf6107735424ff7901614953e7df403`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:15.117519 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-125d8991-3c9f-4690-b62b-3616eb976d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:15.177161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 817/1641] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:15.178507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e94a79-7054-43a4-9e61-6381a0143ea3 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:15.189491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e94a79-7054-43a4-9e61-6381a0143ea3 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:15.194059 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a18195-e5ed-41aa-9f5f-cc6ecaa3ca82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:15.194772 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a18195-e5ed-41aa-9f5f-cc6ecaa3ca82 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:15.196058 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a18195-e5ed-41aa-9f5f-cc6ecaa3ca82 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:15.196447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a18195-e5ed-41aa-9f5f-cc6ecaa3ca82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:15.248977 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 825/1642] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:26:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:15.250978 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d8991-3c9f-4690-b62b-3616eb976d71 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:15.262026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d8991-3c9f-4690-b62b-3616eb976d71 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:15.266810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cf794-4ab9-47d9-901a-714be259d122 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:15.267690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cf794-4ab9-47d9-901a-714be259d122 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:15.267925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cf794-4ab9-47d9-901a-714be259d122 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:15.268187 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2cf794-4ab9-47d9-901a-714be259d122 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:15.282943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2cf794-4ab9-47d9-901a-714be259d122 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:15.284585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2cf794-4ab9-47d9-901a-714be259d122 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:15.310541 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 826/1643] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:15.332440 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a18195-e5ed-41aa-9f5f-cc6ecaa3ca82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:15.333847 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a18195-e5ed-41aa-9f5f-cc6ecaa3ca82 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:15.372568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 818/1644] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:15.408635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cf794-4ab9-47d9-901a-714be259d122 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:15.433253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2cf794-4ab9-47d9-901a-714be259d122 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:15.456467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95016a7-cf5d-4429-8a5e-de15bf2a47c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:15.457546 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95016a7-cf5d-4429-8a5e-de15bf2a47c6 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:15.458058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95016a7-cf5d-4429-8a5e-de15bf2a47c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:15.458546 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95016a7-cf5d-4429-8a5e-de15bf2a47c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:15.486377 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95016a7-cf5d-4429-8a5e-de15bf2a47c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:15.488298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95016a7-cf5d-4429-8a5e-de15bf2a47c6 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:15.509232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 827/1645] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:16.686185 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a18195-e5ed-41aa-9f5f-cc6ecaa3ca82 service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:16.695944 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a18195-e5ed-41aa-9f5f-cc6ecaa3ca82 service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:16.710107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a7c49-b7bb-479d-af67-1ab5f1fb2513 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:16.710622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95016a7-cf5d-4429-8a5e-de15bf2a47c6 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:16.711096 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a7c49-b7bb-479d-af67-1ab5f1fb2513 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:16.711362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a7c49-b7bb-479d-af67-1ab5f1fb2513 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:16.711646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a7c49-b7bb-479d-af67-1ab5f1fb2513 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:16.725896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95016a7-cf5d-4429-8a5e-de15bf2a47c6 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:16.746119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a15d4-cd79-4e6d-93e8-dc3fd68a1b68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:16.746921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a15d4-cd79-4e6d-93e8-dc3fd68a1b68 service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:16.747320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a15d4-cd79-4e6d-93e8-dc3fd68a1b68 service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:16.747660 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a15d4-cd79-4e6d-93e8-dc3fd68a1b68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:16.843769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a7c49-b7bb-479d-af67-1ab5f1fb2513 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:16.845286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a7c49-b7bb-479d-af67-1ab5f1fb2513 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:16.873453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 819/1646] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:16.878709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a15d4-cd79-4e6d-93e8-dc3fd68a1b68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:16.880860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a15d4-cd79-4e6d-93e8-dc3fd68a1b68 service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:16.891706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 828/1647] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:26:17.998598 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a54aa-58b8-475b-9fa9-8ef713991a12 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:17.999522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a54aa-58b8-475b-9fa9-8ef713991a12 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:18.000037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a54aa-58b8-475b-9fa9-8ef713991a12 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:18.002045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 820/1648] 10.222.0.44 () {54 vars in 912 bytes} [Mon May 9 04:26:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:26:18.009241 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa911ebc-552d-43c3-ada6-7fc4cfc35bd3 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:18.010784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa911ebc-552d-43c3-ada6-7fc4cfc35bd3 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:18.011326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa911ebc-552d-43c3-ada6-7fc4cfc35bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:18.063591 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-aa911ebc-552d-43c3-ada6-7fc4cfc35bd3 None None] MFA Rules not processed for user `0d852975a2e84925a45775a16cc52f32`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:18.065357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-aa911ebc-552d-43c3-ada6-7fc4cfc35bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:18.187954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 829/1649] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:26:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:18.312363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a54aa-58b8-475b-9fa9-8ef713991a12 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:18.336338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a54aa-58b8-475b-9fa9-8ef713991a12 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:18.366928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d49b5-d7cb-4f90-89ea-5e31d20f9b90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:18.370857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d49b5-d7cb-4f90-89ea-5e31d20f9b90 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:18.371205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d49b5-d7cb-4f90-89ea-5e31d20f9b90 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:18.371498 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d49b5-d7cb-4f90-89ea-5e31d20f9b90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:18.407145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d49b5-d7cb-4f90-89ea-5e31d20f9b90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:18.410500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d49b5-d7cb-4f90-89ea-5e31d20f9b90 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:18.444878 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 821/1650] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:19.520857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa911ebc-552d-43c3-ada6-7fc4cfc35bd3 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:19.725217 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa911ebc-552d-43c3-ada6-7fc4cfc35bd3 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:19.728735 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:19.729718 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:19.730073 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:19.730444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] PATH_INFO: `/v3/users/8381a286bb624131b5a610de86488e84` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:19.731793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8381a286bb624131b5a610de86488e84)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:19.732933 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: WARNING py.warnings [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 04:26:19.732933 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: warnings.warn(msg) May 09 04:26:19.732933 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: May 09 04:26:19.734222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:19.877248 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8381a286bb624131b5a610de86488e84'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:19.878679 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8381a286bb624131b5a610de86488e84'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:19.902485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8381a286bb624131b5a610de86488e84'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:19.946336 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8381a286bb624131b5a610de86488e84'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:19.947088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8381a286bb624131b5a610de86488e84'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:20.024144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 830/1651] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:19 2022] DELETE /identity/v3/users/8381a286bb624131b5a610de86488e84 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:20.028660 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b40d3-84b8-4323-9f9a-75359e7be747 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:20.029649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b40d3-84b8-4323-9f9a-75359e7be747 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:20.030088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b40d3-84b8-4323-9f9a-75359e7be747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:20.066235 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-662b40d3-84b8-4323-9f9a-75359e7be747 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:20.069950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-662b40d3-84b8-4323-9f9a-75359e7be747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:20.255048 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 822/1652] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:20.263174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:20.264666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:20.272765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b40d3-84b8-4323-9f9a-75359e7be747 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:20.279434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-662b40d3-84b8-4323-9f9a-75359e7be747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:20.297487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae888c7-b7b6-4ab9-b2b3-a2175998ea1c admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:20.323929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e5659-faf6-4861-b2e1-5834b4e54879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:20.325509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e5659-faf6-4861-b2e1-5834b4e54879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:20.327021 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e5659-faf6-4861-b2e1-5834b4e54879 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:20.327451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e5659-faf6-4861-b2e1-5834b4e54879 admin admin] PATH_INFO: `/v3/users/63a787dd74a34eedacf2aea3de2f5c90` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:20.333596 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e5659-faf6-4861-b2e1-5834b4e54879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a787dd74a34eedacf2aea3de2f5c90)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:20.338110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e5659-faf6-4861-b2e1-5834b4e54879 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:20.442251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b34e5659-faf6-4861-b2e1-5834b4e54879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a787dd74a34eedacf2aea3de2f5c90'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:20.444473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b34e5659-faf6-4861-b2e1-5834b4e54879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a787dd74a34eedacf2aea3de2f5c90'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:20.473503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b34e5659-faf6-4861-b2e1-5834b4e54879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a787dd74a34eedacf2aea3de2f5c90'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:20.477135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b40d3-84b8-4323-9f9a-75359e7be747 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:20.487222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b34e5659-faf6-4861-b2e1-5834b4e54879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a787dd74a34eedacf2aea3de2f5c90'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:20.488586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c1235-49f6-44bf-8da4-b689704f9ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:20.489700 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b34e5659-faf6-4861-b2e1-5834b4e54879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a787dd74a34eedacf2aea3de2f5c90'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:20.490577 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c1235-49f6-44bf-8da4-b689704f9ece service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:20.490713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c1235-49f6-44bf-8da4-b689704f9ece service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:20.491120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c1235-49f6-44bf-8da4-b689704f9ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:20.507246 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-aa3c1235-49f6-44bf-8da4-b689704f9ece service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:20.547922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c1235-49f6-44bf-8da4-b689704f9ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:20.549356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c1235-49f6-44bf-8da4-b689704f9ece service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:20.550721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 831/1653] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:20 2022] DELETE /identity/v3/users/63a787dd74a34eedacf2aea3de2f5c90 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:20.557218 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99878838-3e1e-4de3-960d-f5935d14f4cf None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:20.557598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99878838-3e1e-4de3-960d-f5935d14f4cf None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:20.557981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99878838-3e1e-4de3-960d-f5935d14f4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:20.577732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 823/1654] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:20.581705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52e547-89b9-4b87-84fb-c3989b4715ea None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:20.582004 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52e547-89b9-4b87-84fb-c3989b4715ea None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:20.588124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52e547-89b9-4b87-84fb-c3989b4715ea None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:20.594947 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 824/1655] 10.222.0.44 () {54 vars in 960 bytes} [Mon May 9 04:26:20 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:26:20.596372 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf1278-1997-452e-b47b-fd91b38f6e9a None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:20.596666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf1278-1997-452e-b47b-fd91b38f6e9a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:20.596944 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf1278-1997-452e-b47b-fd91b38f6e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:20.608154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-99878838-3e1e-4de3-960d-f5935d14f4cf None None] MFA Rules not processed for user `85429c28c994443eb30b98ece56c4c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:20.609611 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-99878838-3e1e-4de3-960d-f5935d14f4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:20.660281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f4bf1278-1997-452e-b47b-fd91b38f6e9a None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:20.661673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f4bf1278-1997-452e-b47b-fd91b38f6e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:20.737461 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 832/1656] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:20.739259 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c6ded-82aa-4788-b212-dcd216040492 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:20.739646 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c6ded-82aa-4788-b212-dcd216040492 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:20.741622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c6ded-82aa-4788-b212-dcd216040492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:20.768447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-f93c6ded-82aa-4788-b212-dcd216040492 None None] MFA Rules not processed for user `d41e05b7ef954dd596c0134b65434dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:20.770830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f93c6ded-82aa-4788-b212-dcd216040492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:20.839283 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 825/1657] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:20.839283 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf1278-1997-452e-b47b-fd91b38f6e9a None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:20.839625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f4bf1278-1997-452e-b47b-fd91b38f6e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:20.946420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 833/1658] 10.222.0.44 () {58 vars in 1057 bytes} [Mon May 9 04:26:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:20.946420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c6ded-82aa-4788-b212-dcd216040492 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:20.952778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f93c6ded-82aa-4788-b212-dcd216040492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:21.130837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf1278-1997-452e-b47b-fd91b38f6e9a None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:21.132820 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c6ded-82aa-4788-b212-dcd216040492 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:21.145040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56681f-596a-4a57-a7fa-3de513feac95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:21.145885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56681f-596a-4a57-a7fa-3de513feac95 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:21.150132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74c913-2c48-45a7-9217-3f31d8502c0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:21.150787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56681f-596a-4a57-a7fa-3de513feac95 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:21.151195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56681f-596a-4a57-a7fa-3de513feac95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:21.151952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74c913-2c48-45a7-9217-3f31d8502c0b service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:21.152532 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74c913-2c48-45a7-9217-3f31d8502c0b service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:21.152968 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74c913-2c48-45a7-9217-3f31d8502c0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:21.170946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74c913-2c48-45a7-9217-3f31d8502c0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:21.179276 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56681f-596a-4a57-a7fa-3de513feac95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:21.182476 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56681f-596a-4a57-a7fa-3de513feac95 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:21.184520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74c913-2c48-45a7-9217-3f31d8502c0b service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:21.207597 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 826/1659] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:21.208072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 834/1660] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:21.210144 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74c913-2c48-45a7-9217-3f31d8502c0b service nova] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:21.241490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74c913-2c48-45a7-9217-3f31d8502c0b service nova] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:21.255592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb37921-e3e1-4e3b-ae08-e5c07a759e2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd41e05b7ef954dd596c0134b65434dd6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:21.256927 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb37921-e3e1-4e3b-ae08-e5c07a759e2b service placement] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:21.257344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb37921-e3e1-4e3b-ae08-e5c07a759e2b service placement] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:21.257756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb37921-e3e1-4e3b-ae08-e5c07a759e2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:21.259809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-6eb37921-e3e1-4e3b-ae08-e5c07a759e2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:21.276528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb37921-e3e1-4e3b-ae08-e5c07a759e2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:21.278379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb37921-e3e1-4e3b-ae08-e5c07a759e2b service placement] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:21.303251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 827/1661] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:21.537843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56681f-596a-4a57-a7fa-3de513feac95 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:21.539291 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3b56681f-596a-4a57-a7fa-3de513feac95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:21.564249 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56681f-596a-4a57-a7fa-3de513feac95 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:21.580852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:21.583018 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:21.586352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:21.586854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9 admin admin] PATH_INFO: `/v3/projects/d8b020b16f9c43cb9e04a786011069c4` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:21.591810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b020b16f9c43cb9e04a786011069c4)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:21.600534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:21.731819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b020b16f9c43cb9e04a786011069c4', 'request_id': 'req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:21.738145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b020b16f9c43cb9e04a786011069c4', 'request_id': 'req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:21.752879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b020b16f9c43cb9e04a786011069c4', 'request_id': 'req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:21.775267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9 admin admin] The token cache is being invalidate because project d8b020b16f9c43cb9e04a786011069c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:21.775781 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:21.776757 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-31e2c6a9-e33c-4c3b-ba85-61e7768928e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:21.779065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 835/1662] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:21 2022] DELETE /identity/v3/projects/d8b020b16f9c43cb9e04a786011069c4 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:21.784301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6b7f7-5b73-477e-9f16-19e57e94b4c0 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:21.784301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6b7f7-5b73-477e-9f16-19e57e94b4c0 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:21.784565 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6b7f7-5b73-477e-9f16-19e57e94b4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:21.844167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-33f6b7f7-5b73-477e-9f16-19e57e94b4c0 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:21.846797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-33f6b7f7-5b73-477e-9f16-19e57e94b4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:22.148748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 828/1663] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:22.156784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31879c5e-b420-460a-8221-b09ea187061f None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:22.157078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31879c5e-b420-460a-8221-b09ea187061f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:22.157361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31879c5e-b420-460a-8221-b09ea187061f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:22.208501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-31879c5e-b420-460a-8221-b09ea187061f None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:22.218956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-31879c5e-b420-460a-8221-b09ea187061f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:22.260875 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 836/1664] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:22.262043 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6b7f7-5b73-477e-9f16-19e57e94b4c0 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:22.272396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6b7f7-5b73-477e-9f16-19e57e94b4c0 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:22.289462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a98012-f302-4eeb-b095-ef686dc5876f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:22.290397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a98012-f302-4eeb-b095-ef686dc5876f admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:22.290844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a98012-f302-4eeb-b095-ef686dc5876f admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:22.291254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a98012-f302-4eeb-b095-ef686dc5876f admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:22.295280 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a98012-f302-4eeb-b095-ef686dc5876f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:22.295884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a98012-f302-4eeb-b095-ef686dc5876f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:22.298289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a98012-f302-4eeb-b095-ef686dc5876f admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:22.326009 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 829/1665] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:22.330771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31879c5e-b420-460a-8221-b09ea187061f None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:22.349159 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31879c5e-b420-460a-8221-b09ea187061f None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:22.384916 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b562220-3534-4685-bbac-22e2cc52cac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:22.385750 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b562220-3534-4685-bbac-22e2cc52cac0 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:22.388413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b562220-3534-4685-bbac-22e2cc52cac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:22.388709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b562220-3534-4685-bbac-22e2cc52cac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:22.403929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b562220-3534-4685-bbac-22e2cc52cac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:22.409285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b562220-3534-4685-bbac-22e2cc52cac0 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:22.501816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a98012-f302-4eeb-b095-ef686dc5876f admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:22.507349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-72a98012-f302-4eeb-b095-ef686dc5876f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:22.518779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 837/1666] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:22 2022] POST /identity/v3/projects => generated 403 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:22.523545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b562220-3534-4685-bbac-22e2cc52cac0 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:22.546272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a98012-f302-4eeb-b095-ef686dc5876f admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:22.727861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:22.728532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b562220-3534-4685-bbac-22e2cc52cac0 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:22.729336 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:22.730176 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:22.730600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] PATH_INFO: `/v3/projects/ff4a73109fde49f994f89b9df5041a34` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:22.733157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff4a73109fde49f994f89b9df5041a34)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:22.734291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: WARNING py.warnings [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 09 04:26:22.734291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: warnings.warn(msg) May 09 04:26:22.734291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: May 09 04:26:22.735671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:22.747785 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614abf4e-46e0-43bb-aa5d-e8a92103eba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:22.751485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614abf4e-46e0-43bb-aa5d-e8a92103eba1 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:22.751818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614abf4e-46e0-43bb-aa5d-e8a92103eba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:22.752130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614abf4e-46e0-43bb-aa5d-e8a92103eba1 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:22.753383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614abf4e-46e0-43bb-aa5d-e8a92103eba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:22.760302 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614abf4e-46e0-43bb-aa5d-e8a92103eba1 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:22.838475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 838/1667] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:22 2022] POST /identity/v3/users => generated 302 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:22.839243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614abf4e-46e0-43bb-aa5d-e8a92103eba1 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:22.885554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614abf4e-46e0-43bb-aa5d-e8a92103eba1 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:22.962982 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff4a73109fde49f994f89b9df5041a34', 'request_id': 'req-e73fd30d-cbcc-42aa-b82f-fd608b826d71'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:23.029566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff4a73109fde49f994f89b9df5041a34', 'request_id': 'req-e73fd30d-cbcc-42aa-b82f-fd608b826d71'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:23.045182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff4a73109fde49f994f89b9df5041a34', 'request_id': 'req-e73fd30d-cbcc-42aa-b82f-fd608b826d71'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:23.089895 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] The token cache is being invalidate because project ff4a73109fde49f994f89b9df5041a34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:23.108974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:23.117234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775f1ea-19da-4825-8432-6dac144de3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:23.117379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:23.117471 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775f1ea-19da-4825-8432-6dac144de3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:23.117611 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775f1ea-19da-4825-8432-6dac144de3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:23.117889 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775f1ea-19da-4825-8432-6dac144de3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:23.118823 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775f1ea-19da-4825-8432-6dac144de3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:23.120677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775f1ea-19da-4825-8432-6dac144de3d5 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:23.126649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 830/1668] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:22 2022] DELETE /identity/v3/projects/ff4a73109fde49f994f89b9df5041a34 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:23.135064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:23.139172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:23.148971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 839/1669] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:23.155228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775f1ea-19da-4825-8432-6dac144de3d5 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:23.160500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6775f1ea-19da-4825-8432-6dac144de3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:23.181731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fd30d-cbcc-42aa-b82f-fd608b826d71 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:23.185399 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775f1ea-19da-4825-8432-6dac144de3d5 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:23.204591 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bb3ba-9c68-4ec1-8bc8-ff45e608729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:23.205299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bb3ba-9c68-4ec1-8bc8-ff45e608729c admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:23.205584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bb3ba-9c68-4ec1-8bc8-ff45e608729c admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:23.206268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bb3ba-9c68-4ec1-8bc8-ff45e608729c admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:23.206587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bb3ba-9c68-4ec1-8bc8-ff45e608729c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:23.219427 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bb3ba-9c68-4ec1-8bc8-ff45e608729c admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:23.219596 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:23.220417 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:23.220702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:23.220969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] PATH_INFO: `/v3/users/19ee2e5b9b984391be2d622c2718cc4a` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:23.227494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19ee2e5b9b984391be2d622c2718cc4a)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:23.229920 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:23.238644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 840/1670] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:23.258139 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bb3ba-9c68-4ec1-8bc8-ff45e608729c admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:23.277057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bb3ba-9c68-4ec1-8bc8-ff45e608729c admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:23.295735 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0c213-e1f6-408f-928b-a2701dcda139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:23.297364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0c213-e1f6-408f-928b-a2701dcda139 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:23.297640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0c213-e1f6-408f-928b-a2701dcda139 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:23.298049 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0c213-e1f6-408f-928b-a2701dcda139 admin admin] PATH_INFO: `/v3/projects/746671802a2c46cdb37e9b9ac39ff8f6/users/a254d129f3204e258e2eaf6de0f47327/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:23.337020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0c213-e1f6-408f-928b-a2701dcda139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=746671802a2c46cdb37e9b9ac39ff8f6, user_id=a254d129f3204e258e2eaf6de0f47327, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:23.340118 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0c213-e1f6-408f-928b-a2701dcda139 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:23.369174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19ee2e5b9b984391be2d622c2718cc4a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:23.371746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19ee2e5b9b984391be2d622c2718cc4a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:23.373505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19ee2e5b9b984391be2d622c2718cc4a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:23.379832 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 841/1671] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:23 2022] PUT /identity/v3/projects/746671802a2c46cdb37e9b9ac39ff8f6/users/a254d129f3204e258e2eaf6de0f47327/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:26:23.392364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8283bb-aa31-4b6d-96a6-c42ef44bf623 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:23.392661 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8283bb-aa31-4b6d-96a6-c42ef44bf623 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:23.392948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8283bb-aa31-4b6d-96a6-c42ef44bf623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:23.400426 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19ee2e5b9b984391be2d622c2718cc4a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:23.420493 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19ee2e5b9b984391be2d622c2718cc4a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:23.476624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-ca8283bb-aa31-4b6d-96a6-c42ef44bf623 None None] MFA Rules not processed for user `a254d129f3204e258e2eaf6de0f47327`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:23.480013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 831/1672] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:23 2022] DELETE /identity/v3/users/19ee2e5b9b984391be2d622c2718cc4a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:23.482153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ca8283bb-aa31-4b6d-96a6-c42ef44bf623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:23.626883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 842/1673] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:26:23.632423 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:23.633984 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:23.752238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc52423-06a7-4fc1-8117-3891d0cc0615 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:23.764781 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f86732-0e8b-43d4-92a1-4e777768e340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:23.765553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f86732-0e8b-43d4-92a1-4e777768e340 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:23.765851 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f86732-0e8b-43d4-92a1-4e777768e340 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:23.766134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f86732-0e8b-43d4-92a1-4e777768e340 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:23.766863 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f86732-0e8b-43d4-92a1-4e777768e340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:23.768995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f86732-0e8b-43d4-92a1-4e777768e340 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:23.822487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 832/1674] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:23 2022] POST /identity/v3/projects => generated 403 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:23.823278 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8283bb-aa31-4b6d-96a6-c42ef44bf623 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:23.841754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8283bb-aa31-4b6d-96a6-c42ef44bf623 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:23.918037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c34b9-2b6d-417e-ac5a-c81525f25ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:23.918797 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c34b9-2b6d-417e-ac5a-c81525f25ffa admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:23.920946 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c34b9-2b6d-417e-ac5a-c81525f25ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:23.921495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c34b9-2b6d-417e-ac5a-c81525f25ffa admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:23.924221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c34b9-2b6d-417e-ac5a-c81525f25ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:23.927653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c34b9-2b6d-417e-ac5a-c81525f25ffa admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:23.966396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f86732-0e8b-43d4-92a1-4e777768e340 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:23.968690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 843/1675] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:23 2022] POST /identity/v3/users => generated 308 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:26:23.982368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f86732-0e8b-43d4-92a1-4e777768e340 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:23.989712 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fc904-7655-4541-8cb2-c1b4a0fe251d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:23.990592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fc904-7655-4541-8cb2-c1b4a0fe251d admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:23.990939 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fc904-7655-4541-8cb2-c1b4a0fe251d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:23.991548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fc904-7655-4541-8cb2-c1b4a0fe251d admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:23.993106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9fc904-7655-4541-8cb2-c1b4a0fe251d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:23.995020 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9fc904-7655-4541-8cb2-c1b4a0fe251d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:24.003610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c34b9-2b6d-417e-ac5a-c81525f25ffa admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:24.015529 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b51c34b9-2b6d-417e-ac5a-c81525f25ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:24.049824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c34b9-2b6d-417e-ac5a-c81525f25ffa admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:24.057060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 833/1676] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:26:24.066412 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fc904-7655-4541-8cb2-c1b4a0fe251d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:24.072028 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194cdd06-07c1-41cf-b545-0072729eaafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:24.072928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cdd06-07c1-41cf-b545-0072729eaafa admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:24.073207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cdd06-07c1-41cf-b545-0072729eaafa admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:24.075800 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cdd06-07c1-41cf-b545-0072729eaafa admin admin] PATH_INFO: `/v3/projects/8519390d3262470593b9acf3ab8f240a` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:24.080348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cdd06-07c1-41cf-b545-0072729eaafa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8519390d3262470593b9acf3ab8f240a)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:24.087187 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cdd06-07c1-41cf-b545-0072729eaafa admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:24.093646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fc904-7655-4541-8cb2-c1b4a0fe251d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:24.105739 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72429043-4ce9-45f8-85b6-c0b66e22e8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:24.113003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72429043-4ce9-45f8-85b6-c0b66e22e8ca admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:24.113343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72429043-4ce9-45f8-85b6-c0b66e22e8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:24.113628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72429043-4ce9-45f8-85b6-c0b66e22e8ca admin admin] PATH_INFO: `/v3/projects/4302ee08d5d64685a1dc61a7ccc31acf/users/e13382a5022d4e5096e3f988a62e6163/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:24.202510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-194cdd06-07c1-41cf-b545-0072729eaafa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8519390d3262470593b9acf3ab8f240a', 'request_id': 'req-194cdd06-07c1-41cf-b545-0072729eaafa'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:24.224204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-194cdd06-07c1-41cf-b545-0072729eaafa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8519390d3262470593b9acf3ab8f240a', 'request_id': 'req-194cdd06-07c1-41cf-b545-0072729eaafa'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:24.245958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-194cdd06-07c1-41cf-b545-0072729eaafa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8519390d3262470593b9acf3ab8f240a', 'request_id': 'req-194cdd06-07c1-41cf-b545-0072729eaafa'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:24.269363 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-194cdd06-07c1-41cf-b545-0072729eaafa admin admin] The token cache is being invalidate because project 8519390d3262470593b9acf3ab8f240a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:24.269505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-194cdd06-07c1-41cf-b545-0072729eaafa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:24.270495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-194cdd06-07c1-41cf-b545-0072729eaafa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:24.278054 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 844/1677] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:24 2022] DELETE /identity/v3/projects/8519390d3262470593b9acf3ab8f240a => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:24.279617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156582f-8dda-4566-9daf-b6a82bc26f32 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:24.279921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156582f-8dda-4566-9daf-b6a82bc26f32 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:24.283208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156582f-8dda-4566-9daf-b6a82bc26f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:24.327025 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c156582f-8dda-4566-9daf-b6a82bc26f32 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:24.328196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c156582f-8dda-4566-9daf-b6a82bc26f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:24.469943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 845/1678] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:24.480676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69194489-c2e4-4917-bfd6-1fba2e4b6b90 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:24.480962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69194489-c2e4-4917-bfd6-1fba2e4b6b90 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:24.481236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69194489-c2e4-4917-bfd6-1fba2e4b6b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:24.484724 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72429043-4ce9-45f8-85b6-c0b66e22e8ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=4302ee08d5d64685a1dc61a7ccc31acf, user_id=e13382a5022d4e5096e3f988a62e6163, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:24.492718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72429043-4ce9-45f8-85b6-c0b66e22e8ca admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:24.525973 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 834/1679] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:24 2022] PUT /identity/v3/projects/4302ee08d5d64685a1dc61a7ccc31acf/users/e13382a5022d4e5096e3f988a62e6163/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:24.527934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72429043-4ce9-45f8-85b6-c0b66e22e8ca admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:24.533667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-72429043-4ce9-45f8-85b6-c0b66e22e8ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:24.537831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-69194489-c2e4-4917-bfd6-1fba2e4b6b90 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:24.542656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-69194489-c2e4-4917-bfd6-1fba2e4b6b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:24.651281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72429043-4ce9-45f8-85b6-c0b66e22e8ca admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:24.657434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f76cb-d3f8-48b4-9929-99e08951a677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:24.658069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f76cb-d3f8-48b4-9929-99e08951a677 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:24.658359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f76cb-d3f8-48b4-9929-99e08951a677 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:24.659025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f76cb-d3f8-48b4-9929-99e08951a677 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:24.659779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f76cb-d3f8-48b4-9929-99e08951a677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:24.662403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f76cb-d3f8-48b4-9929-99e08951a677 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:24.678904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 835/1680] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:24.680289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f76cb-d3f8-48b4-9929-99e08951a677 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:24.695983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f76cb-d3f8-48b4-9929-99e08951a677 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:24.702567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 846/1681] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:24.703794 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69194489-c2e4-4917-bfd6-1fba2e4b6b90 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:24.705302 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443c3fc-74e0-4dd9-bedf-f935136c6112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:24.706454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443c3fc-74e0-4dd9-bedf-f935136c6112 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:24.706940 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443c3fc-74e0-4dd9-bedf-f935136c6112 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:24.707403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443c3fc-74e0-4dd9-bedf-f935136c6112 admin admin] PATH_INFO: `/v3/projects/4302ee08d5d64685a1dc61a7ccc31acf/users/e13382a5022d4e5096e3f988a62e6163/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:24.710683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443c3fc-74e0-4dd9-bedf-f935136c6112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4302ee08d5d64685a1dc61a7ccc31acf, user_id=e13382a5022d4e5096e3f988a62e6163, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:24.714422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443c3fc-74e0-4dd9-bedf-f935136c6112 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:24.723307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69194489-c2e4-4917-bfd6-1fba2e4b6b90 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:24.738072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 836/1682] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:24 2022] PUT /identity/v3/projects/4302ee08d5d64685a1dc61a7ccc31acf/users/e13382a5022d4e5096e3f988a62e6163/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:26:24.740087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfb630-327a-4901-a8e3-6d22e2405a08 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:24.740578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfb630-327a-4901-a8e3-6d22e2405a08 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:24.741612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfb630-327a-4901-a8e3-6d22e2405a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:24.774191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-c5dfb630-327a-4901-a8e3-6d22e2405a08 None None] MFA Rules not processed for user `e13382a5022d4e5096e3f988a62e6163`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:24.775745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c5dfb630-327a-4901-a8e3-6d22e2405a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:24.879934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 837/1683] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:24 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:24.886710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2f7d6-0be2-47f8-a0b4-8b220d35a92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:24.887724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2f7d6-0be2-47f8-a0b4-8b220d35a92c admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:24.888040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2f7d6-0be2-47f8-a0b4-8b220d35a92c admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:24.888321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2f7d6-0be2-47f8-a0b4-8b220d35a92c admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:24.889026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfb630-327a-4901-a8e3-6d22e2405a08 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:24.889147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2f7d6-0be2-47f8-a0b4-8b220d35a92c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:24.889418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2f7d6-0be2-47f8-a0b4-8b220d35a92c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:24.891086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2f7d6-0be2-47f8-a0b4-8b220d35a92c admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:24.891716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c5dfb630-327a-4901-a8e3-6d22e2405a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:24.927932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 847/1684] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:24.934085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2f7d6-0be2-47f8-a0b4-8b220d35a92c admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:24.946601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2f7d6-0be2-47f8-a0b4-8b220d35a92c admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:24.954347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2a471-317a-4eff-bb53-688327ba1c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:24.955035 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2a471-317a-4eff-bb53-688327ba1c13 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:24.955317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2a471-317a-4eff-bb53-688327ba1c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:24.955594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2a471-317a-4eff-bb53-688327ba1c13 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:24.956538 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2a471-317a-4eff-bb53-688327ba1c13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:24.958743 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2a471-317a-4eff-bb53-688327ba1c13 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:25.010192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 848/1685] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:24 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:25.015242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2a471-317a-4eff-bb53-688327ba1c13 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:25.015548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfb630-327a-4901-a8e3-6d22e2405a08 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:25.176147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0433d3b-4bf0-495b-8018-489cc9fed175 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:25.176841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0433d3b-4bf0-495b-8018-489cc9fed175 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:25.178326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0433d3b-4bf0-495b-8018-489cc9fed175 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:25.178611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0433d3b-4bf0-495b-8018-489cc9fed175 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:25.195587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2a471-317a-4eff-bb53-688327ba1c13 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:25.211331 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fdad84-ae4c-45ec-8483-3153de5db726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:25.212035 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdad84-ae4c-45ec-8483-3153de5db726 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:25.213380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdad84-ae4c-45ec-8483-3153de5db726 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:25.213670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdad84-ae4c-45ec-8483-3153de5db726 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:25.214684 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fdad84-ae4c-45ec-8483-3153de5db726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:25.216657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fdad84-ae4c-45ec-8483-3153de5db726 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:25.273693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 849/1686] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:25 2022] POST /identity/v3/users => generated 290 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:25.283332 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fdad84-ae4c-45ec-8483-3153de5db726 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:25.315291 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fdad84-ae4c-45ec-8483-3153de5db726 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:25.328617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129b1f8-64d0-4ebe-a4d2-56122ca89bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:25.344172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129b1f8-64d0-4ebe-a4d2-56122ca89bfd admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:25.344429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129b1f8-64d0-4ebe-a4d2-56122ca89bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:25.345344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129b1f8-64d0-4ebe-a4d2-56122ca89bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:25.348884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129b1f8-64d0-4ebe-a4d2-56122ca89bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:25.353041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129b1f8-64d0-4ebe-a4d2-56122ca89bfd admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:25.384540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0433d3b-4bf0-495b-8018-489cc9fed175 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:25.394290 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0433d3b-4bf0-495b-8018-489cc9fed175 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:25.403220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 850/1687] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:25.404958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129b1f8-64d0-4ebe-a4d2-56122ca89bfd admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:25.433012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 838/1688] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:24 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:25.435979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129b1f8-64d0-4ebe-a4d2-56122ca89bfd admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:25.469163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab853e3-9f51-47a8-9180-9c45a3e6907f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:25.469860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab853e3-9f51-47a8-9180-9c45a3e6907f admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:25.470144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab853e3-9f51-47a8-9180-9c45a3e6907f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:25.476085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab853e3-9f51-47a8-9180-9c45a3e6907f admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:25.476085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab853e3-9f51-47a8-9180-9c45a3e6907f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:25.476085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab853e3-9f51-47a8-9180-9c45a3e6907f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:25.514754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 851/1689] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:25.516161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0433d3b-4bf0-495b-8018-489cc9fed175 service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:25.550218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0433d3b-4bf0-495b-8018-489cc9fed175 service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:25.595124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df80e2-ff24-457d-b8c6-1c27cb673486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:25.600551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df80e2-ff24-457d-b8c6-1c27cb673486 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:25.600841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df80e2-ff24-457d-b8c6-1c27cb673486 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:25.601121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df80e2-ff24-457d-b8c6-1c27cb673486 admin admin] PATH_INFO: `/v3/projects/70738699d604494abc44cc2619d5e909/users/68d388424db5451b9fe3b1e9a086d53e/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:25.649693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df80e2-ff24-457d-b8c6-1c27cb673486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70738699d604494abc44cc2619d5e909, user_id=68d388424db5451b9fe3b1e9a086d53e, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:25.656453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df80e2-ff24-457d-b8c6-1c27cb673486 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:25.688003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 839/1690] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:25 2022] PUT /identity/v3/projects/70738699d604494abc44cc2619d5e909/users/68d388424db5451b9fe3b1e9a086d53e/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:25.701264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42174e0-84d1-48e3-8155-0956f8d743f1 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:25.702103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42174e0-84d1-48e3-8155-0956f8d743f1 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:25.702426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42174e0-84d1-48e3-8155-0956f8d743f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:25.751539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df80e2-ff24-457d-b8c6-1c27cb673486 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:25.760822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e42174e0-84d1-48e3-8155-0956f8d743f1 None None] MFA Rules not processed for user `68d388424db5451b9fe3b1e9a086d53e`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:25.762913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e42174e0-84d1-48e3-8155-0956f8d743f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:25.922407 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 852/1691] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:25.931075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42174e0-84d1-48e3-8155-0956f8d743f1 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:25.990860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df80e2-ff24-457d-b8c6-1c27cb673486 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:26.004471 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430275b-387e-4be3-ac5d-52b2841a0d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:26.005421 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430275b-387e-4be3-ac5d-52b2841a0d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:26.013589 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430275b-387e-4be3-ac5d-52b2841a0d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:26.014529 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430275b-387e-4be3-ac5d-52b2841a0d10 admin admin] PATH_INFO: `/v3/users/a254d129f3204e258e2eaf6de0f47327` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:26.019996 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430275b-387e-4be3-ac5d-52b2841a0d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a254d129f3204e258e2eaf6de0f47327)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:26.025111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430275b-387e-4be3-ac5d-52b2841a0d10 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:26.106384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42174e0-84d1-48e3-8155-0956f8d743f1 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:26.125289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac54f70-d1d0-4cb0-9988-fce57970a32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:26.126098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac54f70-d1d0-4cb0-9988-fce57970a32c admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:26.126494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac54f70-d1d0-4cb0-9988-fce57970a32c admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:26.126868 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac54f70-d1d0-4cb0-9988-fce57970a32c admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:26.127685 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac54f70-d1d0-4cb0-9988-fce57970a32c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:26.129761 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac54f70-d1d0-4cb0-9988-fce57970a32c admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:26.142062 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4430275b-387e-4be3-ac5d-52b2841a0d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a254d129f3204e258e2eaf6de0f47327'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:26.143257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4430275b-387e-4be3-ac5d-52b2841a0d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a254d129f3204e258e2eaf6de0f47327'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:26.145146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4430275b-387e-4be3-ac5d-52b2841a0d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a254d129f3204e258e2eaf6de0f47327'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:26.167436 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4430275b-387e-4be3-ac5d-52b2841a0d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a254d129f3204e258e2eaf6de0f47327'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:26.180214 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4430275b-387e-4be3-ac5d-52b2841a0d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a254d129f3204e258e2eaf6de0f47327'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:26.203963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 853/1692] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:25 2022] POST /identity/v3/projects => generated 381 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:26.206846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac54f70-d1d0-4cb0-9988-fce57970a32c admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:26.208837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8ac54f70-d1d0-4cb0-9988-fce57970a32c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:26.252930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 840/1693] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:25 2022] DELETE /identity/v3/users/a254d129f3204e258e2eaf6de0f47327 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:26.254913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204e9f-e9cf-4f5a-b13f-ba79480e04b5 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:26.255316 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204e9f-e9cf-4f5a-b13f-ba79480e04b5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:26.256802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204e9f-e9cf-4f5a-b13f-ba79480e04b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:26.294082 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-4c204e9f-e9cf-4f5a-b13f-ba79480e04b5 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:26.297667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-4c204e9f-e9cf-4f5a-b13f-ba79480e04b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:26.391103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac54f70-d1d0-4cb0-9988-fce57970a32c admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:26.446936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 841/1694] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:26.493626 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c204e9f-e9cf-4f5a-b13f-ba79480e04b5 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:26.495087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-4c204e9f-e9cf-4f5a-b13f-ba79480e04b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:26.600040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543243c-5d7f-4e11-a9f4-4e34ff6d6e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:26.600970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543243c-5d7f-4e11-a9f4-4e34ff6d6e4b admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:26.601401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543243c-5d7f-4e11-a9f4-4e34ff6d6e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:26.602382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543243c-5d7f-4e11-a9f4-4e34ff6d6e4b admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:26.603707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543243c-5d7f-4e11-a9f4-4e34ff6d6e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:26.605839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543243c-5d7f-4e11-a9f4-4e34ff6d6e4b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:26.656165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 854/1695] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:26 2022] POST /identity/v3/users => generated 297 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:26:26.659883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543243c-5d7f-4e11-a9f4-4e34ff6d6e4b admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:26.667803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c204e9f-e9cf-4f5a-b13f-ba79480e04b5 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:26.676180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543243c-5d7f-4e11-a9f4-4e34ff6d6e4b admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:26.684943 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34737374-be49-4ab8-9a58-1653852f4d3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:26.691067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34737374-be49-4ab8-9a58-1653852f4d3d service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:26.691067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34737374-be49-4ab8-9a58-1653852f4d3d service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:26.691411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34737374-be49-4ab8-9a58-1653852f4d3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:26.696248 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193420d-63a4-437b-adf9-9035306f6cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:26.696929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193420d-63a4-437b-adf9-9035306f6cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:26.697213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193420d-63a4-437b-adf9-9035306f6cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:26.697478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193420d-63a4-437b-adf9-9035306f6cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:26.698281 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193420d-63a4-437b-adf9-9035306f6cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:26.699811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193420d-63a4-437b-adf9-9035306f6cd3 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:26.725654 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34737374-be49-4ab8-9a58-1653852f4d3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:26.727111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34737374-be49-4ab8-9a58-1653852f4d3d service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:26.758239 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 855/1696] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:26:26.759765 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193420d-63a4-437b-adf9-9035306f6cd3 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:26.778144 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 842/1697] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:26.790262 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193420d-63a4-437b-adf9-9035306f6cd3 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:26.811059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ee058-b6af-4bbd-add1-02c28d651cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:26.812274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ee058-b6af-4bbd-add1-02c28d651cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:26.812557 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ee058-b6af-4bbd-add1-02c28d651cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:26.812830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ee058-b6af-4bbd-add1-02c28d651cbe admin admin] PATH_INFO: `/v3/projects/4a5aa51ce7254e7ab17c0dc67f4ae082/users/c7f525c1df9d4ab8af2ea4d8e36df30f/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:26.858315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ee058-b6af-4bbd-add1-02c28d651cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5aa51ce7254e7ab17c0dc67f4ae082, user_id=c7f525c1df9d4ab8af2ea4d8e36df30f, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:26.861657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ee058-b6af-4bbd-add1-02c28d651cbe admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:26.902197 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 856/1698] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:26 2022] PUT /identity/v3/projects/4a5aa51ce7254e7ab17c0dc67f4ae082/users/c7f525c1df9d4ab8af2ea4d8e36df30f/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:26.906125 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34737374-be49-4ab8-9a58-1653852f4d3d service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:26.923882 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34737374-be49-4ab8-9a58-1653852f4d3d service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:27.097618 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa5390-6989-436e-9cf9-67333d1268d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:27.098746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa5390-6989-436e-9cf9-67333d1268d8 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:27.099848 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa5390-6989-436e-9cf9-67333d1268d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:27.100463 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa5390-6989-436e-9cf9-67333d1268d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:27.103097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa5390-6989-436e-9cf9-67333d1268d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:27.104891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa5390-6989-436e-9cf9-67333d1268d8 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:27.127505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 843/1699] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:27.131996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ee058-b6af-4bbd-add1-02c28d651cbe admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:27.149312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ee058-b6af-4bbd-add1-02c28d651cbe admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:27.164887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b821f09-5cdf-4544-92f3-fedc31c7291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:27.167007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b821f09-5cdf-4544-92f3-fedc31c7291e admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:27.167511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b821f09-5cdf-4544-92f3-fedc31c7291e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:27.168076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b821f09-5cdf-4544-92f3-fedc31c7291e admin admin] PATH_INFO: `/v3/projects/4a5aa51ce7254e7ab17c0dc67f4ae082/users/c7f525c1df9d4ab8af2ea4d8e36df30f/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:27.174189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b821f09-5cdf-4544-92f3-fedc31c7291e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5aa51ce7254e7ab17c0dc67f4ae082, user_id=c7f525c1df9d4ab8af2ea4d8e36df30f, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:27.192823 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b821f09-5cdf-4544-92f3-fedc31c7291e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:27.236204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 857/1700] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:27 2022] PUT /identity/v3/projects/4a5aa51ce7254e7ab17c0dc67f4ae082/users/c7f525c1df9d4ab8af2ea4d8e36df30f/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:27.239107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31bf45-4344-4838-b2e8-791d68723a25 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:27.239405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31bf45-4344-4838-b2e8-791d68723a25 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:27.239752 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31bf45-4344-4838-b2e8-791d68723a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:27.280855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-be31bf45-4344-4838-b2e8-791d68723a25 None None] MFA Rules not processed for user `c7f525c1df9d4ab8af2ea4d8e36df30f`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:27.281691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-be31bf45-4344-4838-b2e8-791d68723a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:27.439565 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 844/1701] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:27.453022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b821f09-5cdf-4544-92f3-fedc31c7291e admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:27.456167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1b821f09-5cdf-4544-92f3-fedc31c7291e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:27.461715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a594e92-d8a1-40ed-bf0e-e23c6b67a1b1 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:27.462008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a594e92-d8a1-40ed-bf0e-e23c6b67a1b1 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:27.462280 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a594e92-d8a1-40ed-bf0e-e23c6b67a1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:27.501861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-7a594e92-d8a1-40ed-bf0e-e23c6b67a1b1 None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:27.507128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-7a594e92-d8a1-40ed-bf0e-e23c6b67a1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:27.652292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b821f09-5cdf-4544-92f3-fedc31c7291e admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:27.667879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a84be-f482-4ce4-a1c3-e163703c5344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:27.667879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a84be-f482-4ce4-a1c3-e163703c5344 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:27.667879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a84be-f482-4ce4-a1c3-e163703c5344 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:27.667879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a84be-f482-4ce4-a1c3-e163703c5344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:27.693046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 845/1702] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:26:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:27.695190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bfe6d-5b62-486a-94ff-9d9f52402b6f None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:27.695486 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bfe6d-5b62-486a-94ff-9d9f52402b6f None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:27.695764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bfe6d-5b62-486a-94ff-9d9f52402b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:27.697617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a84be-f482-4ce4-a1c3-e163703c5344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:27.702218 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a84be-f482-4ce4-a1c3-e163703c5344 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:27.748466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-9c2bfe6d-5b62-486a-94ff-9d9f52402b6f None None] MFA Rules not processed for user `4f8f6c77f898475e8acf1c4042c3928b`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:27.749669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9c2bfe6d-5b62-486a-94ff-9d9f52402b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:27.777455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 858/1703] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:27.778764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a84be-f482-4ce4-a1c3-e163703c5344 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:27.784329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4e8a84be-f482-4ce4-a1c3-e163703c5344 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:27.872119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 846/1704] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:27.892274 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bfe6d-5b62-486a-94ff-9d9f52402b6f None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:27.915295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9c2bfe6d-5b62-486a-94ff-9d9f52402b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:28.014120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a84be-f482-4ce4-a1c3-e163703c5344 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:28.024645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba7489-285f-4641-9f3b-3abcb2f7d682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:28.030944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba7489-285f-4641-9f3b-3abcb2f7d682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:28.031607 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba7489-285f-4641-9f3b-3abcb2f7d682 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:28.032660 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba7489-285f-4641-9f3b-3abcb2f7d682 admin admin] PATH_INFO: `/v3/projects/746671802a2c46cdb37e9b9ac39ff8f6` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:28.036473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba7489-285f-4641-9f3b-3abcb2f7d682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=746671802a2c46cdb37e9b9ac39ff8f6)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:28.040000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba7489-285f-4641-9f3b-3abcb2f7d682 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:28.125780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bfe6d-5b62-486a-94ff-9d9f52402b6f None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:28.144984 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735fe1c-7e55-4b15-a212-d6c9d9c5c71e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:28.145885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735fe1c-7e55-4b15-a212-d6c9d9c5c71e service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:28.146371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735fe1c-7e55-4b15-a212-d6c9d9c5c71e service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:28.146795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735fe1c-7e55-4b15-a212-d6c9d9c5c71e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:28.159006 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735fe1c-7e55-4b15-a212-d6c9d9c5c71e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:28.165606 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735fe1c-7e55-4b15-a212-d6c9d9c5c71e service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:28.166421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-a2ba7489-285f-4641-9f3b-3abcb2f7d682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '746671802a2c46cdb37e9b9ac39ff8f6', 'request_id': 'req-a2ba7489-285f-4641-9f3b-3abcb2f7d682'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:28.180237 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-a2ba7489-285f-4641-9f3b-3abcb2f7d682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '746671802a2c46cdb37e9b9ac39ff8f6', 'request_id': 'req-a2ba7489-285f-4641-9f3b-3abcb2f7d682'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:28.185349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 847/1705] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:28.186448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735fe1c-7e55-4b15-a212-d6c9d9c5c71e service nova] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:28.189173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-6735fe1c-7e55-4b15-a212-d6c9d9c5c71e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:28.206806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-a2ba7489-285f-4641-9f3b-3abcb2f7d682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '746671802a2c46cdb37e9b9ac39ff8f6', 'request_id': 'req-a2ba7489-285f-4641-9f3b-3abcb2f7d682'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:28.235884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-a2ba7489-285f-4641-9f3b-3abcb2f7d682 admin admin] The token cache is being invalidate because project 746671802a2c46cdb37e9b9ac39ff8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:28.241083 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-a2ba7489-285f-4641-9f3b-3abcb2f7d682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:28.242381 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-a2ba7489-285f-4641-9f3b-3abcb2f7d682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:28.257638 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 859/1706] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:27 2022] DELETE /identity/v3/projects/746671802a2c46cdb37e9b9ac39ff8f6 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:28.259080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0194c18-15e5-4a0a-bd33-24a8e804c308 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:28.259372 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0194c18-15e5-4a0a-bd33-24a8e804c308 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:28.259650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0194c18-15e5-4a0a-bd33-24a8e804c308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:28.341282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735fe1c-7e55-4b15-a212-d6c9d9c5c71e service nova] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:28.352693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e0194c18-15e5-4a0a-bd33-24a8e804c308 None None] MFA Rules not processed for user `85429c28c994443eb30b98ece56c4c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:28.353998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e0194c18-15e5-4a0a-bd33-24a8e804c308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:28.472256 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18933f6-e6a1-45f0-9470-0b66fd310f1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:28.472971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18933f6-e6a1-45f0-9470-0b66fd310f1b service swift] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:28.473254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18933f6-e6a1-45f0-9470-0b66fd310f1b service swift] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:28.473570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18933f6-e6a1-45f0-9470-0b66fd310f1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:28.477212 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f18933f6-e6a1-45f0-9470-0b66fd310f1b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:28.558222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 860/1707] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:28.565508 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0194c18-15e5-4a0a-bd33-24a8e804c308 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:28.569496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e0194c18-15e5-4a0a-bd33-24a8e804c308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:28.645760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18933f6-e6a1-45f0-9470-0b66fd310f1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:28.651097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18933f6-e6a1-45f0-9470-0b66fd310f1b service swift] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:28.686560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 848/1708] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:26:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:26:28.687477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18933f6-e6a1-45f0-9470-0b66fd310f1b service swift] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:28.691774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f18933f6-e6a1-45f0-9470-0b66fd310f1b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:28.796072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0194c18-15e5-4a0a-bd33-24a8e804c308 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:28.835700 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:28.839924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:28.840299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:28.840585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] PATH_INFO: `/v3/users/e13382a5022d4e5096e3f988a62e6163` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:28.854009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e13382a5022d4e5096e3f988a62e6163)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:28.854009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:28.976754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18933f6-e6a1-45f0-9470-0b66fd310f1b service swift] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:28.995735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3f365-2487-4418-bd40-64e2afccfe05 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7f525c1df9d4ab8af2ea4d8e36df30f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a5aa51ce7254e7ab17c0dc67f4ae082', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:28.996640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f365-2487-4418-bd40-64e2afccfe05 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:28.997044 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f365-2487-4418-bd40-64e2afccfe05 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:28.997442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f365-2487-4418-bd40-64e2afccfe05 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:28.998325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3f365-2487-4418-bd40-64e2afccfe05 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:28.998798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3f365-2487-4418-bd40-64e2afccfe05 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:29.000946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3f365-2487-4418-bd40-64e2afccfe05 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:29.021966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e13382a5022d4e5096e3f988a62e6163'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.028691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 849/1709] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:29.028919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e13382a5022d4e5096e3f988a62e6163'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.029875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3f365-2487-4418-bd40-64e2afccfe05 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:29.051864 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3f365-2487-4418-bd40-64e2afccfe05 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:29.053292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e13382a5022d4e5096e3f988a62e6163'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.068229 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6801f99-0824-4df6-a6bf-28b02dadb623 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7f525c1df9d4ab8af2ea4d8e36df30f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a5aa51ce7254e7ab17c0dc67f4ae082', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:29.069122 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6801f99-0824-4df6-a6bf-28b02dadb623 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:29.069757 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6801f99-0824-4df6-a6bf-28b02dadb623 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:29.070336 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6801f99-0824-4df6-a6bf-28b02dadb623 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:29.071255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6801f99-0824-4df6-a6bf-28b02dadb623 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:29.078273 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6801f99-0824-4df6-a6bf-28b02dadb623 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:29.101775 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e13382a5022d4e5096e3f988a62e6163'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.101775 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e13382a5022d4e5096e3f988a62e6163'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.165331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 850/1710] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:29 2022] POST /identity/v3/projects => generated 353 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:29.166597 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6801f99-0824-4df6-a6bf-28b02dadb623 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:29.168582 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a6801f99-0824-4df6-a6bf-28b02dadb623 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:29.199771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 861/1711] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:28 2022] DELETE /identity/v3/users/e13382a5022d4e5096e3f988a62e6163 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:29.305669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6801f99-0824-4df6-a6bf-28b02dadb623 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:29.368593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:29.370029 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:29.514194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c605c-a94f-4d6c-b30a-9e4df25c1869 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7f525c1df9d4ab8af2ea4d8e36df30f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a5aa51ce7254e7ab17c0dc67f4ae082', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:29.515210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c605c-a94f-4d6c-b30a-9e4df25c1869 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:29.515521 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c605c-a94f-4d6c-b30a-9e4df25c1869 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:29.515799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c605c-a94f-4d6c-b30a-9e4df25c1869 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:29.516563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c605c-a94f-4d6c-b30a-9e4df25c1869 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:29.516873 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c605c-a94f-4d6c-b30a-9e4df25c1869 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:29.519141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c605c-a94f-4d6c-b30a-9e4df25c1869 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:29.560750 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 851/1712] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:29.563124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f0314-0be2-4e3f-b0b0-078b4fa64ef3 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:29.568435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0431b-b8d8-47d2-9115-0f383a8c0c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:29.569578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0431b-b8d8-47d2-9115-0f383a8c0c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:29.569860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0431b-b8d8-47d2-9115-0f383a8c0c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:29.570133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0431b-b8d8-47d2-9115-0f383a8c0c73 admin admin] PATH_INFO: `/v3/users/3fd3e173caa849b98a2e58339001e0b7` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:29.571376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0431b-b8d8-47d2-9115-0f383a8c0c73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fd3e173caa849b98a2e58339001e0b7)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:29.573418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0431b-b8d8-47d2-9115-0f383a8c0c73 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:29.637218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c605c-a94f-4d6c-b30a-9e4df25c1869 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:29.660874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c605c-a94f-4d6c-b30a-9e4df25c1869 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:29.681329 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7f525c1df9d4ab8af2ea4d8e36df30f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a5aa51ce7254e7ab17c0dc67f4ae082', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:29.682237 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:29.685149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:29.685461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] PATH_INFO: `/v3/projects/d6464bb34f4f4c43b5f8b5c5a1c94fa9` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:29.691586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d6464bb34f4f4c43b5f8b5c5a1c94fa9)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:29.705794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:29.711820 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9cc0431b-b8d8-47d2-9115-0f383a8c0c73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fd3e173caa849b98a2e58339001e0b7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.714137 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9cc0431b-b8d8-47d2-9115-0f383a8c0c73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fd3e173caa849b98a2e58339001e0b7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.728752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9cc0431b-b8d8-47d2-9115-0f383a8c0c73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fd3e173caa849b98a2e58339001e0b7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.754313 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9cc0431b-b8d8-47d2-9115-0f383a8c0c73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fd3e173caa849b98a2e58339001e0b7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.757427 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9cc0431b-b8d8-47d2-9115-0f383a8c0c73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fd3e173caa849b98a2e58339001e0b7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.841335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 862/1713] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:29 2022] DELETE /identity/v3/users/3fd3e173caa849b98a2e58339001e0b7 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:29.846546 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615a6cf-1574-43a8-a16e-9ecd5acbf26d None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:29.847001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615a6cf-1574-43a8-a16e-9ecd5acbf26d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:29.847468 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615a6cf-1574-43a8-a16e-9ecd5acbf26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:29.848414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6464bb34f4f4c43b5f8b5c5a1c94fa9', 'request_id': 'req-1e17a132-ac8d-4198-af12-f3ecd8043e04'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.868131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6464bb34f4f4c43b5f8b5c5a1c94fa9', 'request_id': 'req-1e17a132-ac8d-4198-af12-f3ecd8043e04'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.873756 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2615a6cf-1574-43a8-a16e-9ecd5acbf26d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:29.874597 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2615a6cf-1574-43a8-a16e-9ecd5acbf26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:29.889408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6464bb34f4f4c43b5f8b5c5a1c94fa9', 'request_id': 'req-1e17a132-ac8d-4198-af12-f3ecd8043e04'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.905680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] The token cache is being invalidate because project d6464bb34f4f4c43b5f8b5c5a1c94fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:29.906371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.908614 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:29.922833 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 852/1714] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:29 2022] DELETE /identity/v3/projects/d6464bb34f4f4c43b5f8b5c5a1c94fa9 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:30.018727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 863/1715] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:30.035435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:30.036953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:30.218714 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17a132-ac8d-4198-af12-f3ecd8043e04 tempest-VolumeQuotaClassesTest-1093520139 tempest-VolumeQuotaClassesTest-1093520139-project-admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:30.392861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca38f88-44fa-4675-a307-74315c15cb3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:30.394025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca38f88-44fa-4675-a307-74315c15cb3b service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:30.394469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca38f88-44fa-4675-a307-74315c15cb3b service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:30.396683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca38f88-44fa-4675-a307-74315c15cb3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:30.424914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f29e33-45b0-446c-8827-1048fc11bdce None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:30.425436 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f29e33-45b0-446c-8827-1048fc11bdce None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:30.425898 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f29e33-45b0-446c-8827-1048fc11bdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:30.432427 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca38f88-44fa-4675-a307-74315c15cb3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:30.434040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca38f88-44fa-4675-a307-74315c15cb3b service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:30.458951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 853/1716] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:30.459996 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca38f88-44fa-4675-a307-74315c15cb3b service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:30.477164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8ca38f88-44fa-4675-a307-74315c15cb3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:30.492052 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca38f88-44fa-4675-a307-74315c15cb3b service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:30.498678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:30.499709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:30.500110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:30.500397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] PATH_INFO: `/v3/users/68d388424db5451b9fe3b1e9a086d53e` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:30.501273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-d8f29e33-45b0-446c-8827-1048fc11bdce None None] MFA Rules not processed for user `0d852975a2e84925a45775a16cc52f32`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:30.501646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=68d388424db5451b9fe3b1e9a086d53e)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:30.503578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:30.518154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d8f29e33-45b0-446c-8827-1048fc11bdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:30.558048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 864/1717] 10.222.0.13 () {58 vars in 1015 bytes} [Mon May 9 04:26:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:30.569264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0907bb-150b-48b7-836a-83d073a41a23 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:30.569747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0907bb-150b-48b7-836a-83d073a41a23 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:30.570027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0907bb-150b-48b7-836a-83d073a41a23 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:30.574304 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 865/1718] 10.222.0.44 () {54 vars in 912 bytes} [Mon May 9 04:26:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 04:26:30.598744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0e917-f097-4a22-b8a9-06b3a14fe40e None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:30.598744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0e917-f097-4a22-b8a9-06b3a14fe40e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:30.598744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0e917-f097-4a22-b8a9-06b3a14fe40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:30.651681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68d388424db5451b9fe3b1e9a086d53e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:30.653330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68d388424db5451b9fe3b1e9a086d53e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:30.654512 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68d388424db5451b9fe3b1e9a086d53e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:30.663399 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-f5c0e917-f097-4a22-b8a9-06b3a14fe40e None None] MFA Rules not processed for user `0d852975a2e84925a45775a16cc52f32`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:30.665955 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f5c0e917-f097-4a22-b8a9-06b3a14fe40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:30.683204 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68d388424db5451b9fe3b1e9a086d53e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:30.688297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 866/1719] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:26:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:30.706730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0e917-f097-4a22-b8a9-06b3a14fe40e None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:30.711222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f5c0e917-f097-4a22-b8a9-06b3a14fe40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:30.731054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68d388424db5451b9fe3b1e9a086d53e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:30.737362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0e917-f097-4a22-b8a9-06b3a14fe40e None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:30.758463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218afa60-73c0-4ba1-80b2-def91ccbc357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:30.758463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218afa60-73c0-4ba1-80b2-def91ccbc357 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:30.758463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218afa60-73c0-4ba1-80b2-def91ccbc357 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:30.758463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218afa60-73c0-4ba1-80b2-def91ccbc357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:30.779321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-218afa60-73c0-4ba1-80b2-def91ccbc357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:30.808829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218afa60-73c0-4ba1-80b2-def91ccbc357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:30.810294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218afa60-73c0-4ba1-80b2-def91ccbc357 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:30.831077 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 854/1720] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:30 2022] DELETE /identity/v3/users/68d388424db5451b9fe3b1e9a086d53e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:30.832203 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:30.844250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 867/1721] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:30.845773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9662abd-b1b8-4935-ae92-44056d01a3fe None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:30.846064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9662abd-b1b8-4935-ae92-44056d01a3fe None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:30.846342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9662abd-b1b8-4935-ae92-44056d01a3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:30.846903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:30.901438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-a9662abd-b1b8-4935-ae92-44056d01a3fe None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:30.902676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a9662abd-b1b8-4935-ae92-44056d01a3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:31.103968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 868/1722] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:31.123619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9662abd-b1b8-4935-ae92-44056d01a3fe None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:31.129213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a9662abd-b1b8-4935-ae92-44056d01a3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:31.157820 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9662abd-b1b8-4935-ae92-44056d01a3fe None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:31.185298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2fd6e-d43a-4bfb-8094-724036802acb admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:31.186580 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49a3dc-4851-4c9f-98e7-8c09b26e3c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:31.187263 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49a3dc-4851-4c9f-98e7-8c09b26e3c53 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:31.187550 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49a3dc-4851-4c9f-98e7-8c09b26e3c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:31.187827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49a3dc-4851-4c9f-98e7-8c09b26e3c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:31.207314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738a6d1-7f76-4220-8c9a-6d978a5bcaa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:31.208907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738a6d1-7f76-4220-8c9a-6d978a5bcaa0 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:31.211047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738a6d1-7f76-4220-8c9a-6d978a5bcaa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:31.211407 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738a6d1-7f76-4220-8c9a-6d978a5bcaa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:31.212791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49a3dc-4851-4c9f-98e7-8c09b26e3c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:31.217431 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49a3dc-4851-4c9f-98e7-8c09b26e3c53 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:31.242270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 869/1723] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:31.243672 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49a3dc-4851-4c9f-98e7-8c09b26e3c53 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:31.245361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-7d49a3dc-4851-4c9f-98e7-8c09b26e3c53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:31.252934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738a6d1-7f76-4220-8c9a-6d978a5bcaa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:31.257574 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738a6d1-7f76-4220-8c9a-6d978a5bcaa0 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:31.276140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49a3dc-4851-4c9f-98e7-8c09b26e3c53 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:31.283595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb38022-c635-403c-a755-1d0e8f060e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:31.284739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb38022-c635-403c-a755-1d0e8f060e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:31.285024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb38022-c635-403c-a755-1d0e8f060e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:31.285296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb38022-c635-403c-a755-1d0e8f060e38 admin admin] PATH_INFO: `/v3/projects/4302ee08d5d64685a1dc61a7ccc31acf` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:31.287052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb38022-c635-403c-a755-1d0e8f060e38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4302ee08d5d64685a1dc61a7ccc31acf)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:31.290346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb38022-c635-403c-a755-1d0e8f060e38 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:31.291757 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 855/1724] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:31.400361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9cb38022-c635-403c-a755-1d0e8f060e38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4302ee08d5d64685a1dc61a7ccc31acf', 'request_id': 'req-9cb38022-c635-403c-a755-1d0e8f060e38'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:31.416940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9cb38022-c635-403c-a755-1d0e8f060e38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4302ee08d5d64685a1dc61a7ccc31acf', 'request_id': 'req-9cb38022-c635-403c-a755-1d0e8f060e38'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:31.452541 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9cb38022-c635-403c-a755-1d0e8f060e38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4302ee08d5d64685a1dc61a7ccc31acf', 'request_id': 'req-9cb38022-c635-403c-a755-1d0e8f060e38'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:31.477658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9cb38022-c635-403c-a755-1d0e8f060e38 admin admin] The token cache is being invalidate because project 4302ee08d5d64685a1dc61a7ccc31acf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:31.477971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9cb38022-c635-403c-a755-1d0e8f060e38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:31.479292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9cb38022-c635-403c-a755-1d0e8f060e38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:31.487228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 870/1725] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:31 2022] DELETE /identity/v3/projects/4302ee08d5d64685a1dc61a7ccc31acf => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:31.488587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432124d-c53f-41d3-b98d-d595a68de4cc None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:31.489163 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432124d-c53f-41d3-b98d-d595a68de4cc None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:31.490254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432124d-c53f-41d3-b98d-d595a68de4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:31.546788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-7432124d-c53f-41d3-b98d-d595a68de4cc None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:31.551282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-7432124d-c53f-41d3-b98d-d595a68de4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:31.765852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 856/1726] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:31.771442 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274948f3-8f5a-4d06-b36a-4b0e508aad67 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:31.772142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274948f3-8f5a-4d06-b36a-4b0e508aad67 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:31.772652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274948f3-8f5a-4d06-b36a-4b0e508aad67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:31.798533 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-274948f3-8f5a-4d06-b36a-4b0e508aad67 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:31.806984 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-274948f3-8f5a-4d06-b36a-4b0e508aad67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:31.832635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432124d-c53f-41d3-b98d-d595a68de4cc None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:31.846130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 871/1727] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:31.852021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432124d-c53f-41d3-b98d-d595a68de4cc None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:31.864311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d37a0-d341-4cf2-89f5-2607ea7efa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:31.865789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d37a0-d341-4cf2-89f5-2607ea7efa68 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:31.866089 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d37a0-d341-4cf2-89f5-2607ea7efa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:31.870795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d37a0-d341-4cf2-89f5-2607ea7efa68 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:31.871812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d37a0-d341-4cf2-89f5-2607ea7efa68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:31.872316 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d37a0-d341-4cf2-89f5-2607ea7efa68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:31.873954 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d37a0-d341-4cf2-89f5-2607ea7efa68 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:31.920215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274948f3-8f5a-4d06-b36a-4b0e508aad67 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:31.927753 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 857/1728] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:31.932899 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274948f3-8f5a-4d06-b36a-4b0e508aad67 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:31.953120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3416596-918a-481e-b402-7b611d6a44bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:31.953839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3416596-918a-481e-b402-7b611d6a44bc admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:31.959596 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3416596-918a-481e-b402-7b611d6a44bc admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:31.960203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3416596-918a-481e-b402-7b611d6a44bc admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:31.961221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3416596-918a-481e-b402-7b611d6a44bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:31.968160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3416596-918a-481e-b402-7b611d6a44bc admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:32.078971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d37a0-d341-4cf2-89f5-2607ea7efa68 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:32.092487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d37a0-d341-4cf2-89f5-2607ea7efa68 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:32.094370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 872/1729] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:31 2022] POST /identity/v3/projects => generated 363 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:26:32.265515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b3d1c-e84a-463b-9ff2-4c8ecaed6f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:32.267998 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b3d1c-e84a-463b-9ff2-4c8ecaed6f32 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:32.268549 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b3d1c-e84a-463b-9ff2-4c8ecaed6f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:32.268978 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b3d1c-e84a-463b-9ff2-4c8ecaed6f32 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:32.269889 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b3d1c-e84a-463b-9ff2-4c8ecaed6f32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:32.273245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b3d1c-e84a-463b-9ff2-4c8ecaed6f32 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:32.307555 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 858/1730] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:32 2022] POST /identity/v3/users => generated 282 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:32.312833 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3416596-918a-481e-b402-7b611d6a44bc admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:32.330718 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3416596-918a-481e-b402-7b611d6a44bc admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:32.343373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f11ce3-6322-40a8-9bd2-f1db0a1f2ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:32.360683 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f11ce3-6322-40a8-9bd2-f1db0a1f2ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:32.361224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f11ce3-6322-40a8-9bd2-f1db0a1f2ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:32.361729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f11ce3-6322-40a8-9bd2-f1db0a1f2ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:32.362993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f11ce3-6322-40a8-9bd2-f1db0a1f2ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:32.365015 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f11ce3-6322-40a8-9bd2-f1db0a1f2ed5 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:32.390236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 873/1731] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:32.390697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b3d1c-e84a-463b-9ff2-4c8ecaed6f32 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:32.402236 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b3d1c-e84a-463b-9ff2-4c8ecaed6f32 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:32.416682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe405326-0efe-4cc9-a5d9-f6abc5d80017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:32.417376 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe405326-0efe-4cc9-a5d9-f6abc5d80017 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:32.417661 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe405326-0efe-4cc9-a5d9-f6abc5d80017 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:32.417938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe405326-0efe-4cc9-a5d9-f6abc5d80017 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:32.418924 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe405326-0efe-4cc9-a5d9-f6abc5d80017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:32.435276 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe405326-0efe-4cc9-a5d9-f6abc5d80017 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:32.465964 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f11ce3-6322-40a8-9bd2-f1db0a1f2ed5 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:32.482424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f11ce3-6322-40a8-9bd2-f1db0a1f2ed5 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:32.485698 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 859/1732] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:32.495288 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983cac4-ab41-4c02-b59f-54d16d9d15cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:32.496236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983cac4-ab41-4c02-b59f-54d16d9d15cc admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:32.496517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983cac4-ab41-4c02-b59f-54d16d9d15cc admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:32.496792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983cac4-ab41-4c02-b59f-54d16d9d15cc admin admin] PATH_INFO: `/v3/projects/4e6aaa9b0f9c451098e3dff451d6a5e6/users/cdd7c824dbc441ea901e978977e8e5f7/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:32.534600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983cac4-ab41-4c02-b59f-54d16d9d15cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6aaa9b0f9c451098e3dff451d6a5e6, user_id=cdd7c824dbc441ea901e978977e8e5f7, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:32.549816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983cac4-ab41-4c02-b59f-54d16d9d15cc admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:32.588908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 874/1733] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:32 2022] PUT /identity/v3/projects/4e6aaa9b0f9c451098e3dff451d6a5e6/users/cdd7c824dbc441ea901e978977e8e5f7/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:32.589110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6951c5-62b9-4ee9-a3fa-6b27c25b1e42 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:32.589110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6951c5-62b9-4ee9-a3fa-6b27c25b1e42 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:32.589110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6951c5-62b9-4ee9-a3fa-6b27c25b1e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:32.678412 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-3a6951c5-62b9-4ee9-a3fa-6b27c25b1e42 None None] MFA Rules not processed for user `cdd7c824dbc441ea901e978977e8e5f7`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:32.682705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3a6951c5-62b9-4ee9-a3fa-6b27c25b1e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:32.754312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983cac4-ab41-4c02-b59f-54d16d9d15cc admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:32.761195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4983cac4-ab41-4c02-b59f-54d16d9d15cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:33.009196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 860/1734] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:32 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:26:33.012883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6951c5-62b9-4ee9-a3fa-6b27c25b1e42 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:33.020147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983cac4-ab41-4c02-b59f-54d16d9d15cc admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:33.040106 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:33.043662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:33.043839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:33.045831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] PATH_INFO: `/v3/projects/70738699d604494abc44cc2619d5e909` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:33.051048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70738699d604494abc44cc2619d5e909)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:33.056607 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:33.068056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6951c5-62b9-4ee9-a3fa-6b27c25b1e42 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:33.105594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b1c63-7dc9-4dfd-a52f-14368c0b3aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:33.106291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b1c63-7dc9-4dfd-a52f-14368c0b3aad admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:33.106579 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b1c63-7dc9-4dfd-a52f-14368c0b3aad admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:33.106859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b1c63-7dc9-4dfd-a52f-14368c0b3aad admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:33.107566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b1c63-7dc9-4dfd-a52f-14368c0b3aad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:33.114555 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b1c63-7dc9-4dfd-a52f-14368c0b3aad admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:33.193234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70738699d604494abc44cc2619d5e909', 'request_id': 'req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:33.209501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70738699d604494abc44cc2619d5e909', 'request_id': 'req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:33.227823 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70738699d604494abc44cc2619d5e909', 'request_id': 'req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:33.231309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 861/1735] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:33 2022] POST /identity/v3/projects => generated 365 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:33.235322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b1c63-7dc9-4dfd-a52f-14368c0b3aad admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:33.253281 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] The token cache is being invalidate because project 70738699d604494abc44cc2619d5e909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:33.253679 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:33.254243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:33.256321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 875/1736] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:32 2022] DELETE /identity/v3/projects/70738699d604494abc44cc2619d5e909 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:33.261902 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:33.264372 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:33.410092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cc014-77b2-4206-a62d-9cd694a3e8e1 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:33.465409 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b1c63-7dc9-4dfd-a52f-14368c0b3aad admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:33.491825 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d6e78-bed9-4faf-9bdd-6c22a7a66548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:33.492556 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d6e78-bed9-4faf-9bdd-6c22a7a66548 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:33.492839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d6e78-bed9-4faf-9bdd-6c22a7a66548 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:33.493123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d6e78-bed9-4faf-9bdd-6c22a7a66548 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:33.493814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d6e78-bed9-4faf-9bdd-6c22a7a66548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:33.495740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d6e78-bed9-4faf-9bdd-6c22a7a66548 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:33.554859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 862/1737] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:33 2022] POST /identity/v3/users => generated 289 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:33.560550 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:33.561523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:33.561854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:33.562131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] PATH_INFO: `/v3/users/c7f525c1df9d4ab8af2ea4d8e36df30f` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:33.563526 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7f525c1df9d4ab8af2ea4d8e36df30f)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:33.565997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:33.573372 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d6e78-bed9-4faf-9bdd-6c22a7a66548 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:33.577885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-ed5d6e78-bed9-4faf-9bdd-6c22a7a66548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:33.641680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d6e78-bed9-4faf-9bdd-6c22a7a66548 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:33.648391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7f525c1df9d4ab8af2ea4d8e36df30f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:33.649878 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7f525c1df9d4ab8af2ea4d8e36df30f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:33.666348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7f525c1df9d4ab8af2ea4d8e36df30f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:33.671519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a9f59-62f5-4879-b960-2bfc17d1885e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:33.673899 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a9f59-62f5-4879-b960-2bfc17d1885e admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:33.674180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a9f59-62f5-4879-b960-2bfc17d1885e admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:33.680857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7f525c1df9d4ab8af2ea4d8e36df30f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:33.681644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7f525c1df9d4ab8af2ea4d8e36df30f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:33.686206 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a9f59-62f5-4879-b960-2bfc17d1885e admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:33.693904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a9f59-62f5-4879-b960-2bfc17d1885e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:33.695933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a9f59-62f5-4879-b960-2bfc17d1885e admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:33.723797 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 876/1738] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:33 2022] DELETE /identity/v3/users/c7f525c1df9d4ab8af2ea4d8e36df30f => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:33.765022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 863/1739] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:33.769378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:33.771000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:33.948712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90143690-cd03-4111-983a-525b6a4a6a9e admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:33.965822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443fede-d80e-4b43-a86f-ef9010d4c2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:33.966842 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443fede-d80e-4b43-a86f-ef9010d4c2a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:33.967131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443fede-d80e-4b43-a86f-ef9010d4c2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:33.967489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443fede-d80e-4b43-a86f-ef9010d4c2a8 admin admin] PATH_INFO: `/v3/projects/5ce3caf5ddc54483a48c8c396bf1736f/users/9c26d472d4734bef82c40e488365c260/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:33.998606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443fede-d80e-4b43-a86f-ef9010d4c2a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ce3caf5ddc54483a48c8c396bf1736f, user_id=9c26d472d4734bef82c40e488365c260, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:34.001837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443fede-d80e-4b43-a86f-ef9010d4c2a8 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:34.037339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a9f59-62f5-4879-b960-2bfc17d1885e admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:34.038305 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 877/1740] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:33 2022] PUT /identity/v3/projects/5ce3caf5ddc54483a48c8c396bf1736f/users/9c26d472d4734bef82c40e488365c260/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:34.058853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a9f59-62f5-4879-b960-2bfc17d1885e admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:34.374107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c95be-bfd6-44f2-8915-ba1bcf350f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:34.380901 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c95be-bfd6-44f2-8915-ba1bcf350f7f admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:34.381194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c95be-bfd6-44f2-8915-ba1bcf350f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:34.383858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c95be-bfd6-44f2-8915-ba1bcf350f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:34.385135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c95be-bfd6-44f2-8915-ba1bcf350f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:34.390066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c95be-bfd6-44f2-8915-ba1bcf350f7f admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:34.431764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443fede-d80e-4b43-a86f-ef9010d4c2a8 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:34.445996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443fede-d80e-4b43-a86f-ef9010d4c2a8 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:34.450217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 864/1741] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:26:34.472296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d61ce2-4006-4a55-ae13-cb2559a6ea2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:34.475099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d61ce2-4006-4a55-ae13-cb2559a6ea2c admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:34.475382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d61ce2-4006-4a55-ae13-cb2559a6ea2c admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:34.475851 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d61ce2-4006-4a55-ae13-cb2559a6ea2c admin admin] PATH_INFO: `/v3/projects/5ce3caf5ddc54483a48c8c396bf1736f/users/9c26d472d4734bef82c40e488365c260/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:34.498562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d61ce2-4006-4a55-ae13-cb2559a6ea2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ce3caf5ddc54483a48c8c396bf1736f, user_id=9c26d472d4734bef82c40e488365c260, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:34.531077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d61ce2-4006-4a55-ae13-cb2559a6ea2c admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:34.599822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5085b6e-bf90-4871-a554-08b0c5bad550 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:34.600856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5085b6e-bf90-4871-a554-08b0c5bad550 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:34.601293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5085b6e-bf90-4871-a554-08b0c5bad550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:34.603495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 878/1742] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:34 2022] PUT /identity/v3/projects/5ce3caf5ddc54483a48c8c396bf1736f/users/9c26d472d4734bef82c40e488365c260/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:34.661963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d5085b6e-bf90-4871-a554-08b0c5bad550 None None] MFA Rules not processed for user `9c26d472d4734bef82c40e488365c260`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:34.664039 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d5085b6e-bf90-4871-a554-08b0c5bad550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:34.838055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 865/1743] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:34 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:34.844235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d61ce2-4006-4a55-ae13-cb2559a6ea2c admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:34.846534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-01d61ce2-4006-4a55-ae13-cb2559a6ea2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:35.049886 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d61ce2-4006-4a55-ae13-cb2559a6ea2c admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:35.072478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014ad90-c413-4271-96a2-6ce040452dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:35.077197 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014ad90-c413-4271-96a2-6ce040452dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:35.077555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014ad90-c413-4271-96a2-6ce040452dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:35.079175 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014ad90-c413-4271-96a2-6ce040452dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:35.129276 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014ad90-c413-4271-96a2-6ce040452dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:35.136630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014ad90-c413-4271-96a2-6ce040452dc3 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:35.186088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 879/1744] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:34 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:35.363514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594d1ab-32bd-420c-90fd-babf272e944f None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:35.364031 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594d1ab-32bd-420c-90fd-babf272e944f None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:35.364632 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594d1ab-32bd-420c-90fd-babf272e944f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:35.417298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-8594d1ab-32bd-420c-90fd-babf272e944f None None] MFA Rules not processed for user `87b7c3ec77d4493dac92bb61d0b376be`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:35.419417 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8594d1ab-32bd-420c-90fd-babf272e944f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:35.602255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab93d8-95ac-4e4d-97f0-901cff9fe5e9 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:35.602753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab93d8-95ac-4e4d-97f0-901cff9fe5e9 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:35.603184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab93d8-95ac-4e4d-97f0-901cff9fe5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:35.615706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 866/1745] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:35.636419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-d4ab93d8-95ac-4e4d-97f0-901cff9fe5e9 None None] MFA Rules not processed for user `87b7c3ec77d4493dac92bb61d0b376be`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:35.637785 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d4ab93d8-95ac-4e4d-97f0-901cff9fe5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:35.654222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 880/1746] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:35.660755 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594d1ab-32bd-420c-90fd-babf272e944f None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:35.663538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8594d1ab-32bd-420c-90fd-babf272e944f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:35.702474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab93d8-95ac-4e4d-97f0-901cff9fe5e9 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:35.706064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d4ab93d8-95ac-4e4d-97f0-901cff9fe5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:35.844834 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594d1ab-32bd-420c-90fd-babf272e944f None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:35.873478 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d8d90-1348-4853-8c6e-ee413087e810 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:35.875074 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d8d90-1348-4853-8c6e-ee413087e810 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:35.878510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d8d90-1348-4853-8c6e-ee413087e810 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:35.879174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d8d90-1348-4853-8c6e-ee413087e810 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:35.895152 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d8d90-1348-4853-8c6e-ee413087e810 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:35.898754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d8d90-1348-4853-8c6e-ee413087e810 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:35.915632 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 867/1747] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:36.013131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab93d8-95ac-4e4d-97f0-901cff9fe5e9 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:36.041932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5d717-ead1-4675-9323-ac8357bc1b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:36.041932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5d717-ead1-4675-9323-ac8357bc1b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:36.041932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5d717-ead1-4675-9323-ac8357bc1b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:36.041932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5d717-ead1-4675-9323-ac8357bc1b58 admin admin] PATH_INFO: `/v3/projects/4b289587284a4aba96b85ffec328efff` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:36.042494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5d717-ead1-4675-9323-ac8357bc1b58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b289587284a4aba96b85ffec328efff)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:36.042494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5d717-ead1-4675-9323-ac8357bc1b58 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:36.050308 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5faa1-bda9-430a-b9bd-22cd1c6e6532 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:36.050595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5faa1-bda9-430a-b9bd-22cd1c6e6532 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:36.050867 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5faa1-bda9-430a-b9bd-22cd1c6e6532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:36.087591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-69f5faa1-bda9-430a-b9bd-22cd1c6e6532 None None] MFA Rules not processed for user `4f8f6c77f898475e8acf1c4042c3928b`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:36.089003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-69f5faa1-bda9-430a-b9bd-22cd1c6e6532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:36.211146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-1fb5d717-ead1-4675-9323-ac8357bc1b58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b289587284a4aba96b85ffec328efff', 'request_id': 'req-1fb5d717-ead1-4675-9323-ac8357bc1b58'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:36.225374 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-1fb5d717-ead1-4675-9323-ac8357bc1b58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b289587284a4aba96b85ffec328efff', 'request_id': 'req-1fb5d717-ead1-4675-9323-ac8357bc1b58'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:36.239378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-1fb5d717-ead1-4675-9323-ac8357bc1b58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b289587284a4aba96b85ffec328efff', 'request_id': 'req-1fb5d717-ead1-4675-9323-ac8357bc1b58'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:36.256830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-1fb5d717-ead1-4675-9323-ac8357bc1b58 admin admin] The token cache is being invalidate because project 4b289587284a4aba96b85ffec328efff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:36.257156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-1fb5d717-ead1-4675-9323-ac8357bc1b58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:36.257857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-1fb5d717-ead1-4675-9323-ac8357bc1b58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:36.260109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 881/1748] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:35 2022] DELETE /identity/v3/projects/4b289587284a4aba96b85ffec328efff => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:36.264791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e248056-0c05-4bbb-9e36-c1e9c4a54e14 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:36.265334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e248056-0c05-4bbb-9e36-c1e9c4a54e14 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:36.265824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e248056-0c05-4bbb-9e36-c1e9c4a54e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:36.294951 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-1e248056-0c05-4bbb-9e36-c1e9c4a54e14 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:36.298208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1e248056-0c05-4bbb-9e36-c1e9c4a54e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:36.343934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 868/1749] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:36.546306 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 882/1750] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:26:36.552332 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9d765-4f07-478b-bfa8-16b5b84cb7af None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:36.558272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9d765-4f07-478b-bfa8-16b5b84cb7af None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:36.558708 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9d765-4f07-478b-bfa8-16b5b84cb7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:36.597701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-5df9d765-4f07-478b-bfa8-16b5b84cb7af None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:36.599307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5df9d765-4f07-478b-bfa8-16b5b84cb7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:36.612650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e248056-0c05-4bbb-9e36-c1e9c4a54e14 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:36.615015 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1e248056-0c05-4bbb-9e36-c1e9c4a54e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:36.641175 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 869/1751] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:26:36.644688 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e248056-0c05-4bbb-9e36-c1e9c4a54e14 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:36.645489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9d765-4f07-478b-bfa8-16b5b84cb7af None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:36.672758 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d224fd6-1ebd-416f-aea9-5a59d7d80ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:36.673771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d224fd6-1ebd-416f-aea9-5a59d7d80ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:36.674384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d224fd6-1ebd-416f-aea9-5a59d7d80ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:36.676978 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9d765-4f07-478b-bfa8-16b5b84cb7af None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:36.678130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d224fd6-1ebd-416f-aea9-5a59d7d80ef8 admin admin] PATH_INFO: `/v3/users/cdd7c824dbc441ea901e978977e8e5f7` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:36.680467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d224fd6-1ebd-416f-aea9-5a59d7d80ef8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdd7c824dbc441ea901e978977e8e5f7)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:36.682453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d224fd6-1ebd-416f-aea9-5a59d7d80ef8 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:36.691471 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e5b66-94dc-4c60-bf36-c1558e167e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:36.693197 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e5b66-94dc-4c60-bf36-c1558e167e71 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:36.694362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e5b66-94dc-4c60-bf36-c1558e167e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:36.694803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e5b66-94dc-4c60-bf36-c1558e167e71 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:36.700237 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e5b66-94dc-4c60-bf36-c1558e167e71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:36.700771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e5b66-94dc-4c60-bf36-c1558e167e71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:36.702710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e5b66-94dc-4c60-bf36-c1558e167e71 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:36.729527 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 870/1752] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:26:36.734005 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e5b66-94dc-4c60-bf36-c1558e167e71 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:36.775554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e5b66-94dc-4c60-bf36-c1558e167e71 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:36.788912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c44b4a-bf3a-436a-82ed-2ddf4f496b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:36.790027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c44b4a-bf3a-436a-82ed-2ddf4f496b19 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:36.790312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c44b4a-bf3a-436a-82ed-2ddf4f496b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:36.790857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c44b4a-bf3a-436a-82ed-2ddf4f496b19 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:36.792121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c44b4a-bf3a-436a-82ed-2ddf4f496b19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:36.794039 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c44b4a-bf3a-436a-82ed-2ddf4f496b19 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:36.795319 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-5d224fd6-1ebd-416f-aea9-5a59d7d80ef8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdd7c824dbc441ea901e978977e8e5f7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:36.797288 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-5d224fd6-1ebd-416f-aea9-5a59d7d80ef8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdd7c824dbc441ea901e978977e8e5f7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:36.818458 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-5d224fd6-1ebd-416f-aea9-5a59d7d80ef8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdd7c824dbc441ea901e978977e8e5f7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:36.837087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-5d224fd6-1ebd-416f-aea9-5a59d7d80ef8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdd7c824dbc441ea901e978977e8e5f7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:36.839706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-5d224fd6-1ebd-416f-aea9-5a59d7d80ef8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdd7c824dbc441ea901e978977e8e5f7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:36.888986 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 871/1753] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:36 2022] POST /identity/v3/projects => generated 391 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:36.897474 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c44b4a-bf3a-436a-82ed-2ddf4f496b19 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:36.902389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f8c44b4a-bf3a-436a-82ed-2ddf4f496b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:36.914954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 883/1754] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:36 2022] DELETE /identity/v3/users/cdd7c824dbc441ea901e978977e8e5f7 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:36.921729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3884db7-1c9b-4088-b30f-5ec8430bd6be None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:36.924961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3884db7-1c9b-4088-b30f-5ec8430bd6be None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:36.926111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3884db7-1c9b-4088-b30f-5ec8430bd6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:37.000483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c3884db7-1c9b-4088-b30f-5ec8430bd6be None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:37.002763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c3884db7-1c9b-4088-b30f-5ec8430bd6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:37.183576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c44b4a-bf3a-436a-82ed-2ddf4f496b19 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:37.405244 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f2714-6d4f-4de5-a342-3439f9c7c319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:37.405837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f2714-6d4f-4de5-a342-3439f9c7c319 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:37.406129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f2714-6d4f-4de5-a342-3439f9c7c319 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:37.406407 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f2714-6d4f-4de5-a342-3439f9c7c319 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:37.407253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f2714-6d4f-4de5-a342-3439f9c7c319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:37.409516 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f2714-6d4f-4de5-a342-3439f9c7c319 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:37.458117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 872/1755] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:36 2022] POST /identity/v3/users => generated 296 bytes in 563 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:37.466809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f2714-6d4f-4de5-a342-3439f9c7c319 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:37.474481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-296f2714-6d4f-4de5-a342-3439f9c7c319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:37.488003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 884/1756] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 568 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:37.491857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3884db7-1c9b-4088-b30f-5ec8430bd6be None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:37.516687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f2714-6d4f-4de5-a342-3439f9c7c319 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:37.519710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3884db7-1c9b-4088-b30f-5ec8430bd6be None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:37.530178 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:37.531064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:37.531344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:37.531689 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] PATH_INFO: `/v3/projects/4a5aa51ce7254e7ab17c0dc67f4ae082` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:37.533417 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5aa51ce7254e7ab17c0dc67f4ae082)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:37.538009 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:37.538666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36e767-bdb6-4c03-940a-8c83c9a7ce93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:37.539470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36e767-bdb6-4c03-940a-8c83c9a7ce93 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:37.546425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36e767-bdb6-4c03-940a-8c83c9a7ce93 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:37.546854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36e767-bdb6-4c03-940a-8c83c9a7ce93 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:37.547741 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36e767-bdb6-4c03-940a-8c83c9a7ce93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:37.551365 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36e767-bdb6-4c03-940a-8c83c9a7ce93 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:37.610853 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 885/1757] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:37.611819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36e767-bdb6-4c03-940a-8c83c9a7ce93 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:37.614338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6c36e767-bdb6-4c03-940a-8c83c9a7ce93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:37.646489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5aa51ce7254e7ab17c0dc67f4ae082', 'request_id': 'req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:37.661371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5aa51ce7254e7ab17c0dc67f4ae082', 'request_id': 'req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:37.673302 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5aa51ce7254e7ab17c0dc67f4ae082', 'request_id': 'req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:37.711826 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] The token cache is being invalidate because project 4a5aa51ce7254e7ab17c0dc67f4ae082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:37.713021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:37.714206 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:37.726221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 873/1758] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:37 2022] DELETE /identity/v3/projects/4a5aa51ce7254e7ab17c0dc67f4ae082 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:37.731223 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:37.741654 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:37.776191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36e767-bdb6-4c03-940a-8c83c9a7ce93 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:37.920664 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54e5b1-4e1a-4ab5-9068-c7f4e2a70ad4 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:37.934220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e3844-adff-493d-a9d4-4b3005377abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:37.936777 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e3844-adff-493d-a9d4-4b3005377abe admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:37.939973 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e3844-adff-493d-a9d4-4b3005377abe admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:37.941066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e3844-adff-493d-a9d4-4b3005377abe admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:37.941963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e3844-adff-493d-a9d4-4b3005377abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:37.947960 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e3844-adff-493d-a9d4-4b3005377abe admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:37.951713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f9755-fffb-405e-8c38-b06dd9bbe80c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:37.952598 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f9755-fffb-405e-8c38-b06dd9bbe80c service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:37.953149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f9755-fffb-405e-8c38-b06dd9bbe80c service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:37.953437 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f9755-fffb-405e-8c38-b06dd9bbe80c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:37.970468 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-dd6f9755-fffb-405e-8c38-b06dd9bbe80c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:38.001923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f9755-fffb-405e-8c38-b06dd9bbe80c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:38.003536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 874/1759] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:38.007069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a55e41-ba5d-4c09-817a-a1b33cbff555 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:38.008475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a55e41-ba5d-4c09-817a-a1b33cbff555 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:38.008715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a55e41-ba5d-4c09-817a-a1b33cbff555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:38.018180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f9755-fffb-405e-8c38-b06dd9bbe80c service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:38.039831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 886/1760] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:38.042377 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f9755-fffb-405e-8c38-b06dd9bbe80c service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:38.067203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f9755-fffb-405e-8c38-b06dd9bbe80c service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:38.075112 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283ef57-f47a-4895-bb10-50a6ccd47a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:38.076000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283ef57-f47a-4895-bb10-50a6ccd47a03 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:38.076288 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283ef57-f47a-4895-bb10-50a6ccd47a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:38.077318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283ef57-f47a-4895-bb10-50a6ccd47a03 admin admin] PATH_INFO: `/v3/projects/776ca5a21dd2425a95ebd514efbabee7/users/9cda7278efbe4e37af96ca1e32888a06/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:38.080191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-09a55e41-ba5d-4c09-817a-a1b33cbff555 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:38.094353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-09a55e41-ba5d-4c09-817a-a1b33cbff555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:38.137339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283ef57-f47a-4895-bb10-50a6ccd47a03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=776ca5a21dd2425a95ebd514efbabee7, user_id=9cda7278efbe4e37af96ca1e32888a06, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:38.145827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283ef57-f47a-4895-bb10-50a6ccd47a03 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:38.169011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 875/1761] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:38.171393 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a47af-2065-43d0-8b70-c95fd85c429c None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:38.171808 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a47af-2065-43d0-8b70-c95fd85c429c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:38.172169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a47af-2065-43d0-8b70-c95fd85c429c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:38.188646 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 887/1762] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:38 2022] PUT /identity/v3/projects/776ca5a21dd2425a95ebd514efbabee7/users/9cda7278efbe4e37af96ca1e32888a06/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:38.200879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497b033-19c7-4ab7-bb6c-5733851d2b4e None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:38.201186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497b033-19c7-4ab7-bb6c-5733851d2b4e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:38.201301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-8b3a47af-2065-43d0-8b70-c95fd85c429c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:38.201467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497b033-19c7-4ab7-bb6c-5733851d2b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:38.237951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8b3a47af-2065-43d0-8b70-c95fd85c429c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:38.259101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-8497b033-19c7-4ab7-bb6c-5733851d2b4e None None] MFA Rules not processed for user `9cda7278efbe4e37af96ca1e32888a06`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:38.260391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8497b033-19c7-4ab7-bb6c-5733851d2b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:38.378328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 876/1763] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:38.385078 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a47af-2065-43d0-8b70-c95fd85c429c None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:38.403860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a47af-2065-43d0-8b70-c95fd85c429c None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:38.419990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277974aa-0e34-44d0-bbbb-901b333dd40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:38.420688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277974aa-0e34-44d0-bbbb-901b333dd40c admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:38.420965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277974aa-0e34-44d0-bbbb-901b333dd40c admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:38.421239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277974aa-0e34-44d0-bbbb-901b333dd40c admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:38.421977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277974aa-0e34-44d0-bbbb-901b333dd40c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:38.422281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277974aa-0e34-44d0-bbbb-901b333dd40c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:38.423989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277974aa-0e34-44d0-bbbb-901b333dd40c admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:38.462458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 877/1764] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:38.467055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277974aa-0e34-44d0-bbbb-901b333dd40c admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:38.481241 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277974aa-0e34-44d0-bbbb-901b333dd40c admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:38.496542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95d36f-95bd-4170-a4f3-2da3ba260cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:38.500350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95d36f-95bd-4170-a4f3-2da3ba260cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:38.500350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95d36f-95bd-4170-a4f3-2da3ba260cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:38.500350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95d36f-95bd-4170-a4f3-2da3ba260cc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:38.500350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95d36f-95bd-4170-a4f3-2da3ba260cc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:38.500749 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95d36f-95bd-4170-a4f3-2da3ba260cc1 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:38.501444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 888/1765] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:26:38.502723 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497b033-19c7-4ab7-bb6c-5733851d2b4e None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:38.564181 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497b033-19c7-4ab7-bb6c-5733851d2b4e None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:38.574105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 878/1766] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:38 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:26:38.575140 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95d36f-95bd-4170-a4f3-2da3ba260cc1 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:38.579629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bc660-f1df-4659-b1ac-09c59b445e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:38.580561 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bc660-f1df-4659-b1ac-09c59b445e49 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:38.581001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bc660-f1df-4659-b1ac-09c59b445e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:38.581414 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bc660-f1df-4659-b1ac-09c59b445e49 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:38.582279 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bc660-f1df-4659-b1ac-09c59b445e49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:38.584974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bc660-f1df-4659-b1ac-09c59b445e49 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:38.708231 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 889/1767] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:38 2022] POST /identity/v3/projects => generated 391 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:38.717165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fb3a8-eb8b-41b9-85af-4dc8fd0246d6 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:38.717457 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fb3a8-eb8b-41b9-85af-4dc8fd0246d6 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:38.717736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fb3a8-eb8b-41b9-85af-4dc8fd0246d6 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:38.722578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 890/1768] 10.222.0.44 () {54 vars in 921 bytes} [Mon May 9 04:26:38 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:26:38.723427 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fb3a8-eb8b-41b9-85af-4dc8fd0246d6 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:38.750075 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95d36f-95bd-4170-a4f3-2da3ba260cc1 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:38.944915 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce51b9-b93b-440f-9aa2-39e428867e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:38.947843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce51b9-b93b-440f-9aa2-39e428867e31 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:38.948312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce51b9-b93b-440f-9aa2-39e428867e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:38.948832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce51b9-b93b-440f-9aa2-39e428867e31 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:38.949680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce51b9-b93b-440f-9aa2-39e428867e31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:38.956538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce51b9-b93b-440f-9aa2-39e428867e31 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:39.024591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 879/1769] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:38 2022] POST /identity/v3/users => generated 291 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:39.028326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd4500-51c3-44b2-9ef7-95e95fb03f87 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:39.028647 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd4500-51c3-44b2-9ef7-95e95fb03f87 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:39.030069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd4500-51c3-44b2-9ef7-95e95fb03f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:39.062532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fb3a8-eb8b-41b9-85af-4dc8fd0246d6 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:39.082956 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-39dd4500-51c3-44b2-9ef7-95e95fb03f87 None None] MFA Rules not processed for user `faf6107735424ff7901614953e7df403`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:39.087087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3635f75-7d41-4575-9fba-125ff44d3327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:39.087783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3635f75-7d41-4575-9fba-125ff44d3327 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:39.088269 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3635f75-7d41-4575-9fba-125ff44d3327 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:39.088494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3635f75-7d41-4575-9fba-125ff44d3327 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:39.089175 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3635f75-7d41-4575-9fba-125ff44d3327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:39.091196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3635f75-7d41-4575-9fba-125ff44d3327 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:39.095357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-39dd4500-51c3-44b2-9ef7-95e95fb03f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:39.159391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 891/1770] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:38 2022] POST /identity/v3/users => generated 302 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:39.160385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3635f75-7d41-4575-9fba-125ff44d3327 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:39.193743 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3635f75-7d41-4575-9fba-125ff44d3327 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:39.212655 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb256cc-aa3b-44d2-a1ff-cae9043c9f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:39.213799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb256cc-aa3b-44d2-a1ff-cae9043c9f1b admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:39.213799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb256cc-aa3b-44d2-a1ff-cae9043c9f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:39.214001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb256cc-aa3b-44d2-a1ff-cae9043c9f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:39.214914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb256cc-aa3b-44d2-a1ff-cae9043c9f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:39.221361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb256cc-aa3b-44d2-a1ff-cae9043c9f1b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:39.258798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 892/1771] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:39.261161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb256cc-aa3b-44d2-a1ff-cae9043c9f1b admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:39.306974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb256cc-aa3b-44d2-a1ff-cae9043c9f1b admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:39.322537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5031fd-fc62-4fc7-b191-de570d49336f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:39.323321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5031fd-fc62-4fc7-b191-de570d49336f admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:39.323554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5031fd-fc62-4fc7-b191-de570d49336f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:39.323829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5031fd-fc62-4fc7-b191-de570d49336f admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:39.324648 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5031fd-fc62-4fc7-b191-de570d49336f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:39.337571 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 880/1772] 10.222.0.44 () {58 vars in 1018 bytes} [Mon May 9 04:26:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:39.339197 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd4500-51c3-44b2-9ef7-95e95fb03f87 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:39.340878 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5031fd-fc62-4fc7-b191-de570d49336f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:39.387735 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 893/1773] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:39.389395 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5031fd-fc62-4fc7-b191-de570d49336f admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:39.398358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd4500-51c3-44b2-9ef7-95e95fb03f87 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:39.416164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d187a-6c3a-4d01-9316-0f13b63276de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:39.418569 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d187a-6c3a-4d01-9316-0f13b63276de admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:39.419151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d187a-6c3a-4d01-9316-0f13b63276de admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:39.420723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d187a-6c3a-4d01-9316-0f13b63276de admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:39.424901 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d187a-6c3a-4d01-9316-0f13b63276de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:39.430934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d187a-6c3a-4d01-9316-0f13b63276de admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:39.437394 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5031fd-fc62-4fc7-b191-de570d49336f admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:39.457708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c49cc-b797-476c-b3e3-af7ead8b7208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:39.460137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 881/1774] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:39.464667 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c49cc-b797-476c-b3e3-af7ead8b7208 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:39.464974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c49cc-b797-476c-b3e3-af7ead8b7208 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:39.469049 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c49cc-b797-476c-b3e3-af7ead8b7208 admin admin] PATH_INFO: `/v3/projects/c8591337c62d4ea29d08891e0e016ce6/users/0ff9f97260e54571aa749fa07524e5fd/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:39.474797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d187a-6c3a-4d01-9316-0f13b63276de admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:39.504152 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d187a-6c3a-4d01-9316-0f13b63276de admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:39.516835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087ba82-5bb2-4cca-92f9-e8904d90fe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:39.518897 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087ba82-5bb2-4cca-92f9-e8904d90fe0a admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:39.520273 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087ba82-5bb2-4cca-92f9-e8904d90fe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:39.523196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087ba82-5bb2-4cca-92f9-e8904d90fe0a admin admin] PATH_INFO: `/v3/projects/0dd9e10217e64a2a9bd5673a803ac71f/users/ea5874eef71e43b7abed80569cd766ac/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:39.559922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c49cc-b797-476c-b3e3-af7ead8b7208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8591337c62d4ea29d08891e0e016ce6, user_id=0ff9f97260e54571aa749fa07524e5fd, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:39.567334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087ba82-5bb2-4cca-92f9-e8904d90fe0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd9e10217e64a2a9bd5673a803ac71f, user_id=ea5874eef71e43b7abed80569cd766ac, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:39.570843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c49cc-b797-476c-b3e3-af7ead8b7208 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:39.571043 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087ba82-5bb2-4cca-92f9-e8904d90fe0a admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:39.596508 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 882/1775] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:39 2022] PUT /identity/v3/projects/0dd9e10217e64a2a9bd5673a803ac71f/users/ea5874eef71e43b7abed80569cd766ac/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:39.598103 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087ba82-5bb2-4cca-92f9-e8904d90fe0a admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:39.600072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-2087ba82-5bb2-4cca-92f9-e8904d90fe0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:39.635281 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 894/1776] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:39 2022] PUT /identity/v3/projects/c8591337c62d4ea29d08891e0e016ce6/users/0ff9f97260e54571aa749fa07524e5fd/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:39.652153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f3b2a-f5aa-42c7-a37e-dd0aeee9172a None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:39.652441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f3b2a-f5aa-42c7-a37e-dd0aeee9172a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:39.652722 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f3b2a-f5aa-42c7-a37e-dd0aeee9172a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:39.767100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-f52f3b2a-f5aa-42c7-a37e-dd0aeee9172a None None] MFA Rules not processed for user `ea5874eef71e43b7abed80569cd766ac`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:39.770370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f52f3b2a-f5aa-42c7-a37e-dd0aeee9172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:39.788799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087ba82-5bb2-4cca-92f9-e8904d90fe0a admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:40.056603 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 895/1777] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:40.071953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f3b2a-f5aa-42c7-a37e-dd0aeee9172a None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:40.114997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a71ea-47d8-456f-a3c4-43e56e587c17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:40.117723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a71ea-47d8-456f-a3c4-43e56e587c17 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:40.118240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a71ea-47d8-456f-a3c4-43e56e587c17 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:40.119489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a71ea-47d8-456f-a3c4-43e56e587c17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:40.155669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a71ea-47d8-456f-a3c4-43e56e587c17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:40.168889 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a71ea-47d8-456f-a3c4-43e56e587c17 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:40.247628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 883/1778] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:40.248949 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a71ea-47d8-456f-a3c4-43e56e587c17 service nova] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:40.400555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f3b2a-f5aa-42c7-a37e-dd0aeee9172a None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:40.419209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff11dd-96c9-49b7-a6a8-2f812719bd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:40.422176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff11dd-96c9-49b7-a6a8-2f812719bd00 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:40.422464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff11dd-96c9-49b7-a6a8-2f812719bd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:40.422741 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff11dd-96c9-49b7-a6a8-2f812719bd00 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:40.423506 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff11dd-96c9-49b7-a6a8-2f812719bd00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:40.431877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff11dd-96c9-49b7-a6a8-2f812719bd00 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:40.467655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a71ea-47d8-456f-a3c4-43e56e587c17 service nova] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:40.493635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59ce33-73f8-4a30-88de-df39f0d01a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:40.494706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59ce33-73f8-4a30-88de-df39f0d01a47 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:40.496053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59ce33-73f8-4a30-88de-df39f0d01a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:40.496563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59ce33-73f8-4a30-88de-df39f0d01a47 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:40.497481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59ce33-73f8-4a30-88de-df39f0d01a47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:40.499612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59ce33-73f8-4a30-88de-df39f0d01a47 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:40.506860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 896/1779] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:40.507842 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff11dd-96c9-49b7-a6a8-2f812719bd00 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:40.511694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-daff11dd-96c9-49b7-a6a8-2f812719bd00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:40.581446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff11dd-96c9-49b7-a6a8-2f812719bd00 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:40.612996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:40.616322 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:40.619318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:40.621957 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 884/1780] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:40 2022] POST /identity/v3/projects => generated 383 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:40.626137 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] PATH_INFO: `/v3/projects/4e6aaa9b0f9c451098e3dff451d6a5e6` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:40.628343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59ce33-73f8-4a30-88de-df39f0d01a47 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:40.633819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e6aaa9b0f9c451098e3dff451d6a5e6)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:40.645179 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:40.888909 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e6aaa9b0f9c451098e3dff451d6a5e6', 'request_id': 'req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:40.925567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e6aaa9b0f9c451098e3dff451d6a5e6', 'request_id': 'req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:40.951188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e6aaa9b0f9c451098e3dff451d6a5e6', 'request_id': 'req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:40.965479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59ce33-73f8-4a30-88de-df39f0d01a47 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:40.975287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] The token cache is being invalidate because project 4e6aaa9b0f9c451098e3dff451d6a5e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:40.975968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:40.978100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:40.993745 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 897/1781] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:40 2022] DELETE /identity/v3/projects/4e6aaa9b0f9c451098e3dff451d6a5e6 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:40.995427 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:41.001017 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:41.241974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cba64f-8a5e-499a-a655-628eec706a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:41.242294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d0e24-fb89-4cca-a6e8-a35b7b23b66b admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:41.246937 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cba64f-8a5e-499a-a655-628eec706a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:41.249547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cba64f-8a5e-499a-a655-628eec706a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:41.250170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cba64f-8a5e-499a-a655-628eec706a0a admin admin] PATH_INFO: `/v3/projects/c8591337c62d4ea29d08891e0e016ce6/users/0ff9f97260e54571aa749fa07524e5fd/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:41.254018 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17193ff-0a06-41f7-9124-ae98ea3f5172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:41.256389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cba64f-8a5e-499a-a655-628eec706a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8591337c62d4ea29d08891e0e016ce6, user_id=0ff9f97260e54571aa749fa07524e5fd, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:41.264259 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17193ff-0a06-41f7-9124-ae98ea3f5172 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:41.264862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17193ff-0a06-41f7-9124-ae98ea3f5172 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:41.265286 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17193ff-0a06-41f7-9124-ae98ea3f5172 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:41.266049 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17193ff-0a06-41f7-9124-ae98ea3f5172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:41.272940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17193ff-0a06-41f7-9124-ae98ea3f5172 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:41.275773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cba64f-8a5e-499a-a655-628eec706a0a admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:41.314960 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 885/1782] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:40 2022] PUT /identity/v3/projects/c8591337c62d4ea29d08891e0e016ce6/users/0ff9f97260e54571aa749fa07524e5fd/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:41.316850 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cba64f-8a5e-499a-a655-628eec706a0a admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:41.327149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-43cba64f-8a5e-499a-a655-628eec706a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:41.350688 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 898/1783] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:40 2022] POST /identity/v3/users => generated 298 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:41.355067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb626eb-7214-4d16-8a13-1df071a0c4b8 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:41.363966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb626eb-7214-4d16-8a13-1df071a0c4b8 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:41.364352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb626eb-7214-4d16-8a13-1df071a0c4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:41.481516 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-0bb626eb-7214-4d16-8a13-1df071a0c4b8 None None] MFA Rules not processed for user `0ff9f97260e54571aa749fa07524e5fd`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:41.483057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-0bb626eb-7214-4d16-8a13-1df071a0c4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:41.563057 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cba64f-8a5e-499a-a655-628eec706a0a admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:41.584581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:41.585740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:41.587833 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:41.588183 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] PATH_INFO: `/v3/users/9c26d472d4734bef82c40e488365c260` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:41.597683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c26d472d4734bef82c40e488365c260)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:41.603493 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:41.805596 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 899/1784] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:41.808216 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb626eb-7214-4d16-8a13-1df071a0c4b8 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:41.842507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb626eb-7214-4d16-8a13-1df071a0c4b8 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:41.860976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c26d472d4734bef82c40e488365c260'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:41.861448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605cffc-a67d-4bc5-bb06-81dfea25f58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:41.862493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605cffc-a67d-4bc5-bb06-81dfea25f58d admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:41.862924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605cffc-a67d-4bc5-bb06-81dfea25f58d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:41.863367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605cffc-a67d-4bc5-bb06-81dfea25f58d admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:41.864128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c26d472d4734bef82c40e488365c260'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:41.864883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605cffc-a67d-4bc5-bb06-81dfea25f58d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:41.866834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605cffc-a67d-4bc5-bb06-81dfea25f58d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:41.867130 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c26d472d4734bef82c40e488365c260'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:41.883552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c26d472d4734bef82c40e488365c260'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:41.900426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 900/1785] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:41.901583 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605cffc-a67d-4bc5-bb06-81dfea25f58d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:41.903767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-9605cffc-a67d-4bc5-bb06-81dfea25f58d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:41.928944 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c26d472d4734bef82c40e488365c260'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:41.974469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605cffc-a67d-4bc5-bb06-81dfea25f58d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:41.978643 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a0293-e2f0-4e94-9e4e-c095e3f9e197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:41.979359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a0293-e2f0-4e94-9e4e-c095e3f9e197 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:41.979587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a0293-e2f0-4e94-9e4e-c095e3f9e197 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:41.979860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a0293-e2f0-4e94-9e4e-c095e3f9e197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:41.982488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-fc9a0293-e2f0-4e94-9e4e-c095e3f9e197 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:41.992173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 886/1786] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:41 2022] DELETE /identity/v3/users/9c26d472d4734bef82c40e488365c260 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:41.993069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:42.000758 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:42.007487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a0293-e2f0-4e94-9e4e-c095e3f9e197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:42.009072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a0293-e2f0-4e94-9e4e-c095e3f9e197 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:42.025122 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 901/1787] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:42.197926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a0293-e2f0-4e94-9e4e-c095e3f9e197 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:42.215725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a0293-e2f0-4e94-9e4e-c095e3f9e197 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:42.268958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc24fd1-e6ff-40c4-8285-d7b25b518ec2 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:42.281353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22237e-a91f-494f-b469-d03854ff4b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:42.285395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22237e-a91f-494f-b469-d03854ff4b4d admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:42.285710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22237e-a91f-494f-b469-d03854ff4b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:42.285989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22237e-a91f-494f-b469-d03854ff4b4d admin admin] PATH_INFO: `/v3/projects/325adb362e0545a8ae52bdc4a6faea3e/users/e6bd465b458146f48bd290ffe35743ee/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:42.310162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22237e-a91f-494f-b469-d03854ff4b4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=325adb362e0545a8ae52bdc4a6faea3e, user_id=e6bd465b458146f48bd290ffe35743ee, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:42.313144 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22237e-a91f-494f-b469-d03854ff4b4d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:42.341983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 887/1788] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:41 2022] PUT /identity/v3/projects/325adb362e0545a8ae52bdc4a6faea3e/users/e6bd465b458146f48bd290ffe35743ee/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:42.345164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b95388-6fa8-43eb-8b62-6d91cab66ab3 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:42.347279 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b95388-6fa8-43eb-8b62-6d91cab66ab3 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:42.347564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b95388-6fa8-43eb-8b62-6d91cab66ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:42.410134 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d7b95388-6fa8-43eb-8b62-6d91cab66ab3 None None] MFA Rules not processed for user `87b7c3ec77d4493dac92bb61d0b376be`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:42.411858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d7b95388-6fa8-43eb-8b62-6d91cab66ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:42.467475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae635e20-5b24-4218-834b-31747d6634ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:42.470382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae635e20-5b24-4218-834b-31747d6634ac service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:42.472196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae635e20-5b24-4218-834b-31747d6634ac service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:42.472803 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae635e20-5b24-4218-834b-31747d6634ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:42.484767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ae635e20-5b24-4218-834b-31747d6634ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:42.572791 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 888/1789] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:42.574962 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b95388-6fa8-43eb-8b62-6d91cab66ab3 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:42.797130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae635e20-5b24-4218-834b-31747d6634ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:42.803856 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae635e20-5b24-4218-834b-31747d6634ac service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:42.833472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b95388-6fa8-43eb-8b62-6d91cab66ab3 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:42.839588 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa087c1-c2b2-4e36-a19b-30cbe7e3097f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:42.839803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa087c1-c2b2-4e36-a19b-30cbe7e3097f admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:42.840157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa087c1-c2b2-4e36-a19b-30cbe7e3097f admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:42.840439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa087c1-c2b2-4e36-a19b-30cbe7e3097f admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:42.841721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa087c1-c2b2-4e36-a19b-30cbe7e3097f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:42.843521 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa087c1-c2b2-4e36-a19b-30cbe7e3097f admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:42.857748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 889/1790] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:42.872215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa087c1-c2b2-4e36-a19b-30cbe7e3097f admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:42.886563 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 902/1791] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:42.930469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa087c1-c2b2-4e36-a19b-30cbe7e3097f admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:42.943913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c1aad-2766-4815-bb2c-2c25fe6154c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:42.944960 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c1aad-2766-4815-bb2c-2c25fe6154c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:42.949338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c1aad-2766-4815-bb2c-2c25fe6154c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:42.949898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c1aad-2766-4815-bb2c-2c25fe6154c9 admin admin] PATH_INFO: `/v3/projects/325adb362e0545a8ae52bdc4a6faea3e/users/e6bd465b458146f48bd290ffe35743ee/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:42.964712 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c1aad-2766-4815-bb2c-2c25fe6154c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=325adb362e0545a8ae52bdc4a6faea3e, user_id=e6bd465b458146f48bd290ffe35743ee, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:42.967646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c1aad-2766-4815-bb2c-2c25fe6154c9 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:42.997250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9afb8-f92c-468d-8367-fd8eb7b8d788 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:42.997506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 890/1792] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:42 2022] PUT /identity/v3/projects/325adb362e0545a8ae52bdc4a6faea3e/users/e6bd465b458146f48bd290ffe35743ee/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:42.997976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9afb8-f92c-468d-8367-fd8eb7b8d788 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:42.997976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9afb8-f92c-468d-8367-fd8eb7b8d788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:43.102441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-12b9afb8-f92c-468d-8367-fd8eb7b8d788 None None] MFA Rules not processed for user `e6bd465b458146f48bd290ffe35743ee`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:43.103686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-12b9afb8-f92c-468d-8367-fd8eb7b8d788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:43.329503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 903/1793] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:43.408079 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c1aad-2766-4815-bb2c-2c25fe6154c9 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:43.410071 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-ce8c1aad-2766-4815-bb2c-2c25fe6154c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:43.505785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c1aad-2766-4815-bb2c-2c25fe6154c9 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:43.510328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38c648-ef3a-4dcb-b23e-e3b9ff505b5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:43.511193 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38c648-ef3a-4dcb-b23e-e3b9ff505b5c service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:43.511673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38c648-ef3a-4dcb-b23e-e3b9ff505b5c service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:43.512132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38c648-ef3a-4dcb-b23e-e3b9ff505b5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:43.558854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38c648-ef3a-4dcb-b23e-e3b9ff505b5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:43.569059 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38c648-ef3a-4dcb-b23e-e3b9ff505b5c service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:43.592527 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 891/1794] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:43.862993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9afb8-f92c-468d-8367-fd8eb7b8d788 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:43.865350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-12b9afb8-f92c-468d-8367-fd8eb7b8d788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:44.092698 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38c648-ef3a-4dcb-b23e-e3b9ff505b5c service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:44.105235 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38c648-ef3a-4dcb-b23e-e3b9ff505b5c service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:44.123912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76c0b8-c5f2-47fb-8c95-a3ed7bf665b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:44.126313 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76c0b8-c5f2-47fb-8c95-a3ed7bf665b6 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:44.126608 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76c0b8-c5f2-47fb-8c95-a3ed7bf665b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:44.126894 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76c0b8-c5f2-47fb-8c95-a3ed7bf665b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:44.134082 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5d76c0b8-c5f2-47fb-8c95-a3ed7bf665b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:44.168631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9afb8-f92c-468d-8367-fd8eb7b8d788 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:44.189953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:44.191940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:44.196762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:44.197283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d admin admin] PATH_INFO: `/v3/projects/5ce3caf5ddc54483a48c8c396bf1736f` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:44.199010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ce3caf5ddc54483a48c8c396bf1736f)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:44.207382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:44.353681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76c0b8-c5f2-47fb-8c95-a3ed7bf665b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:44.358172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76c0b8-c5f2-47fb-8c95-a3ed7bf665b6 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:44.381130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ce3caf5ddc54483a48c8c396bf1736f', 'request_id': 'req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:44.397639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 892/1795] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:44.418557 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ce3caf5ddc54483a48c8c396bf1736f', 'request_id': 'req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:44.442079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ce3caf5ddc54483a48c8c396bf1736f', 'request_id': 'req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:44.467255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d admin admin] The token cache is being invalidate because project 5ce3caf5ddc54483a48c8c396bf1736f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:44.467873 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:44.472352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6f0d07f3-7a2a-4091-b1cc-4a44b1f6451d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:44.476568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 904/1796] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:43 2022] DELETE /identity/v3/projects/5ce3caf5ddc54483a48c8c396bf1736f => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:44.482750 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb203f-9eeb-43ae-8274-8af2f46bf1ef None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:44.483255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb203f-9eeb-43ae-8274-8af2f46bf1ef None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:44.484078 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb203f-9eeb-43ae-8274-8af2f46bf1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:44.557574 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-8cbb203f-9eeb-43ae-8274-8af2f46bf1ef None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:44.562331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8cbb203f-9eeb-43ae-8274-8af2f46bf1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:44.772684 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 893/1797] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:44.774396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee75c5-03ae-4bf4-93a1-f6d0f15b8f33 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:44.774893 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee75c5-03ae-4bf4-93a1-f6d0f15b8f33 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:44.775328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee75c5-03ae-4bf4-93a1-f6d0f15b8f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:44.832244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-8eee75c5-03ae-4bf4-93a1-f6d0f15b8f33 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:44.834178 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8eee75c5-03ae-4bf4-93a1-f6d0f15b8f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:44.847217 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 905/1798] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:44.851763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb203f-9eeb-43ae-8274-8af2f46bf1ef None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:44.870244 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb203f-9eeb-43ae-8274-8af2f46bf1ef None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:44.879470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a691060-9bed-4996-bf41-6df8d6d7ccec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:44.885665 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a691060-9bed-4996-bf41-6df8d6d7ccec admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:44.886040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a691060-9bed-4996-bf41-6df8d6d7ccec admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:44.886338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a691060-9bed-4996-bf41-6df8d6d7ccec admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:44.887289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a691060-9bed-4996-bf41-6df8d6d7ccec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:44.887607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a691060-9bed-4996-bf41-6df8d6d7ccec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:44.889587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a691060-9bed-4996-bf41-6df8d6d7ccec admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:44.922223 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 894/1799] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:44.926246 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee75c5-03ae-4bf4-93a1-f6d0f15b8f33 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:44.942224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee75c5-03ae-4bf4-93a1-f6d0f15b8f33 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:44.955702 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3b697-0bee-4d7a-b781-bbf6b03efa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:44.956542 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3b697-0bee-4d7a-b781-bbf6b03efa61 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:44.956977 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3b697-0bee-4d7a-b781-bbf6b03efa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:44.957209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3b697-0bee-4d7a-b781-bbf6b03efa61 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:44.957965 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3b697-0bee-4d7a-b781-bbf6b03efa61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:44.960699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3b697-0bee-4d7a-b781-bbf6b03efa61 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:45.048632 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 906/1800] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:44 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:45.060066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a691060-9bed-4996-bf41-6df8d6d7ccec admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:45.085148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a691060-9bed-4996-bf41-6df8d6d7ccec admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:45.245859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0af834-26d0-46a0-ae7f-58820c3b5de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:45.247153 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0af834-26d0-46a0-ae7f-58820c3b5de8 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:45.247787 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0af834-26d0-46a0-ae7f-58820c3b5de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:45.248785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0af834-26d0-46a0-ae7f-58820c3b5de8 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:45.250427 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0af834-26d0-46a0-ae7f-58820c3b5de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:45.263614 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0af834-26d0-46a0-ae7f-58820c3b5de8 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:45.298266 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 895/1801] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:45 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:45.299497 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3b697-0bee-4d7a-b781-bbf6b03efa61 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:45.313746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3b697-0bee-4d7a-b781-bbf6b03efa61 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:45.319325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c988c4d-3d3a-4f0f-b021-76313497af06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:45.320181 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c988c4d-3d3a-4f0f-b021-76313497af06 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:45.320766 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c988c4d-3d3a-4f0f-b021-76313497af06 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:45.321439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c988c4d-3d3a-4f0f-b021-76313497af06 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:45.322630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c988c4d-3d3a-4f0f-b021-76313497af06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:45.325345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c988c4d-3d3a-4f0f-b021-76313497af06 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:45.344871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 907/1802] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:45.353322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0af834-26d0-46a0-ae7f-58820c3b5de8 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:45.370588 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0af834-26d0-46a0-ae7f-58820c3b5de8 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:45.392742 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8abaf1-b2c6-49cd-b8da-988b9d9516a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:45.393505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8abaf1-b2c6-49cd-b8da-988b9d9516a2 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:45.394254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8abaf1-b2c6-49cd-b8da-988b9d9516a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:45.395067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8abaf1-b2c6-49cd-b8da-988b9d9516a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:45.396965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8abaf1-b2c6-49cd-b8da-988b9d9516a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:45.398777 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8abaf1-b2c6-49cd-b8da-988b9d9516a2 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:45.422914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 896/1803] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:26:45.427878 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c988c4d-3d3a-4f0f-b021-76313497af06 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:45.444636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c988c4d-3d3a-4f0f-b021-76313497af06 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:45.449420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147640c-3bfe-4900-8dd9-64ea4bd6de10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:45.450509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147640c-3bfe-4900-8dd9-64ea4bd6de10 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:45.450957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147640c-3bfe-4900-8dd9-64ea4bd6de10 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:45.451387 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147640c-3bfe-4900-8dd9-64ea4bd6de10 admin admin] PATH_INFO: `/v3/projects/191d26a45a4a4aa5b0847a717c8007f3/users/5c29159f83644a51a86604fa1df2950a/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:45.470370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147640c-3bfe-4900-8dd9-64ea4bd6de10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=191d26a45a4a4aa5b0847a717c8007f3, user_id=5c29159f83644a51a86604fa1df2950a, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:45.474972 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147640c-3bfe-4900-8dd9-64ea4bd6de10 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:45.524135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 908/1804] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:45 2022] PUT /identity/v3/projects/191d26a45a4a4aa5b0847a717c8007f3/users/5c29159f83644a51a86604fa1df2950a/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:45.532494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b72024-92e6-4524-b4aa-86737ba037ce None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:45.532886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b72024-92e6-4524-b4aa-86737ba037ce None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:45.533529 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b72024-92e6-4524-b4aa-86737ba037ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:45.598048 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-29b72024-92e6-4524-b4aa-86737ba037ce None None] MFA Rules not processed for user `5c29159f83644a51a86604fa1df2950a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:45.606277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-29b72024-92e6-4524-b4aa-86737ba037ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:45.763476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 897/1805] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:45.774990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147640c-3bfe-4900-8dd9-64ea4bd6de10 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:45.973366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147640c-3bfe-4900-8dd9-64ea4bd6de10 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:45.984639 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1ad1c-249a-4f35-a68a-e8fb9cf946e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:45.985320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1ad1c-249a-4f35-a68a-e8fb9cf946e2 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:45.985600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1ad1c-249a-4f35-a68a-e8fb9cf946e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:45.985967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1ad1c-249a-4f35-a68a-e8fb9cf946e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:45.986572 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e1ad1c-249a-4f35-a68a-e8fb9cf946e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:45.991563 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e1ad1c-249a-4f35-a68a-e8fb9cf946e2 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:46.071091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 909/1806] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:45 2022] POST /identity/v3/projects => generated 375 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:46.075107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b72024-92e6-4524-b4aa-86737ba037ce None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:46.089551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b72024-92e6-4524-b4aa-86737ba037ce None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:46.241761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5479c-2559-41c9-b9c1-f5082346186d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:46.242437 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5479c-2559-41c9-b9c1-f5082346186d admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:46.242769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5479c-2559-41c9-b9c1-f5082346186d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:46.242992 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5479c-2559-41c9-b9c1-f5082346186d admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:46.243681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5479c-2559-41c9-b9c1-f5082346186d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:46.245779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5479c-2559-41c9-b9c1-f5082346186d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:46.291931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 898/1807] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:46 2022] POST /identity/v3/users => generated 294 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:46.295676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1ad1c-249a-4f35-a68a-e8fb9cf946e2 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:46.310277 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1ad1c-249a-4f35-a68a-e8fb9cf946e2 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:46.320553 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8cf25-751f-4b5f-b52a-56775c265b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:46.321242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8cf25-751f-4b5f-b52a-56775c265b35 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:46.321524 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8cf25-751f-4b5f-b52a-56775c265b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:46.321799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8cf25-751f-4b5f-b52a-56775c265b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:46.322615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8cf25-751f-4b5f-b52a-56775c265b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:46.324188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8cf25-751f-4b5f-b52a-56775c265b35 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:46.336606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 910/1808] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:26:46.340522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5479c-2559-41c9-b9c1-f5082346186d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:46.354309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5479c-2559-41c9-b9c1-f5082346186d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:46.364904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1931a4-3c7f-483f-bdcf-af8704218c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:46.366452 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1931a4-3c7f-483f-bdcf-af8704218c48 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:46.366732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1931a4-3c7f-483f-bdcf-af8704218c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:46.367364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1931a4-3c7f-483f-bdcf-af8704218c48 admin admin] PATH_INFO: `/v3/projects/89c1466b7a194c91bcd166f02d33a822/users/9d45a430b3a4425ca7773c5da77b8b6a/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:46.394752 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1931a4-3c7f-483f-bdcf-af8704218c48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c1466b7a194c91bcd166f02d33a822, user_id=9d45a430b3a4425ca7773c5da77b8b6a, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:46.398338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1931a4-3c7f-483f-bdcf-af8704218c48 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:46.412957 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 899/1809] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:46 2022] PUT /identity/v3/projects/89c1466b7a194c91bcd166f02d33a822/users/9d45a430b3a4425ca7773c5da77b8b6a/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:26:46.417062 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8cf25-751f-4b5f-b52a-56775c265b35 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:46.428713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8cf25-751f-4b5f-b52a-56775c265b35 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:46.486803 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a540ee6-e047-4e92-bc01-6d80011a1fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:46.487488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a540ee6-e047-4e92-bc01-6d80011a1fbd admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:46.487776 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a540ee6-e047-4e92-bc01-6d80011a1fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:46.488083 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a540ee6-e047-4e92-bc01-6d80011a1fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:46.488817 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a540ee6-e047-4e92-bc01-6d80011a1fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:46.490682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a540ee6-e047-4e92-bc01-6d80011a1fbd admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:46.500394 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 911/1810] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:46.504727 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1931a4-3c7f-483f-bdcf-af8704218c48 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:46.515734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1931a4-3c7f-483f-bdcf-af8704218c48 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:46.520421 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b430b7e-df26-4dcc-bdc8-b3574c5f3059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:46.521936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b430b7e-df26-4dcc-bdc8-b3574c5f3059 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:46.522214 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b430b7e-df26-4dcc-bdc8-b3574c5f3059 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:46.522491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b430b7e-df26-4dcc-bdc8-b3574c5f3059 admin admin] PATH_INFO: `/v3/projects/89c1466b7a194c91bcd166f02d33a822/users/9d45a430b3a4425ca7773c5da77b8b6a/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:46.524525 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b430b7e-df26-4dcc-bdc8-b3574c5f3059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c1466b7a194c91bcd166f02d33a822, user_id=9d45a430b3a4425ca7773c5da77b8b6a, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:46.527137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b430b7e-df26-4dcc-bdc8-b3574c5f3059 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:46.543553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 900/1811] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:46 2022] PUT /identity/v3/projects/89c1466b7a194c91bcd166f02d33a822/users/9d45a430b3a4425ca7773c5da77b8b6a/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:46.554724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d3f32-d1b8-4cc6-9ee1-8ce0538cd279 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:46.555038 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d3f32-d1b8-4cc6-9ee1-8ce0538cd279 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:46.555298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d3f32-d1b8-4cc6-9ee1-8ce0538cd279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:46.584038 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-673d3f32-d1b8-4cc6-9ee1-8ce0538cd279 None None] MFA Rules not processed for user `9d45a430b3a4425ca7773c5da77b8b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:46.585194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-673d3f32-d1b8-4cc6-9ee1-8ce0538cd279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:46.652351 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 912/1812] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:46.663186 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b430b7e-df26-4dcc-bdc8-b3574c5f3059 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:46.664878 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0b430b7e-df26-4dcc-bdc8-b3574c5f3059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:46.730115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b430b7e-df26-4dcc-bdc8-b3574c5f3059 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:46.733804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdeceb8-fa46-495e-a0f0-f3bdce5f4304 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:46.734461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdeceb8-fa46-495e-a0f0-f3bdce5f4304 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:46.734752 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdeceb8-fa46-495e-a0f0-f3bdce5f4304 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:46.735031 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdeceb8-fa46-495e-a0f0-f3bdce5f4304 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:46.789057 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdeceb8-fa46-495e-a0f0-f3bdce5f4304 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:46.790481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdeceb8-fa46-495e-a0f0-f3bdce5f4304 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:46.798961 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 901/1813] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:49.000447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7211ea3-b649-43ee-a874-14f99875d123 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:49.000941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7211ea3-b649-43ee-a874-14f99875d123 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:49.000941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7211ea3-b649-43ee-a874-14f99875d123 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:49.002311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 913/1814] 10.222.0.44 () {54 vars in 921 bytes} [Mon May 9 04:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:26:49.022630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d02d7-968e-419b-a6b4-cd164e03c294 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:49.023171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d02d7-968e-419b-a6b4-cd164e03c294 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:49.023399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d02d7-968e-419b-a6b4-cd164e03c294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:49.042852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-196d02d7-968e-419b-a6b4-cd164e03c294 None None] MFA Rules not processed for user `faf6107735424ff7901614953e7df403`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:49.044127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-196d02d7-968e-419b-a6b4-cd164e03c294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:49.126796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 902/1815] 10.222.0.44 () {58 vars in 1018 bytes} [Mon May 9 04:26:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:49.140473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7211ea3-b649-43ee-a874-14f99875d123 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:49.142284 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b7211ea3-b649-43ee-a874-14f99875d123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:49.159605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7211ea3-b649-43ee-a874-14f99875d123 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:49.165005 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b907fc3-24d7-4f46-9332-5f7386a552ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:49.165647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b907fc3-24d7-4f46-9332-5f7386a552ab service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:49.165927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b907fc3-24d7-4f46-9332-5f7386a552ab service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:49.166197 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b907fc3-24d7-4f46-9332-5f7386a552ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:49.174712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b907fc3-24d7-4f46-9332-5f7386a552ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:49.176300 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b907fc3-24d7-4f46-9332-5f7386a552ab service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:49.187626 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 914/1816] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:51.819850 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d02d7-968e-419b-a6b4-cd164e03c294 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:51.821712 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-196d02d7-968e-419b-a6b4-cd164e03c294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:51.901846 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d02d7-968e-419b-a6b4-cd164e03c294 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:51.906332 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:51.907206 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:51.907490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:51.907764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] PATH_INFO: `/v3/users/9cda7278efbe4e37af96ca1e32888a06` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:51.908979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cda7278efbe4e37af96ca1e32888a06)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:51.910876 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:51.957686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cda7278efbe4e37af96ca1e32888a06'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:51.958666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cda7278efbe4e37af96ca1e32888a06'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:51.959337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cda7278efbe4e37af96ca1e32888a06'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:51.967113 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cda7278efbe4e37af96ca1e32888a06'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:51.978878 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cda7278efbe4e37af96ca1e32888a06'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:51.996461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 903/1817] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:51 2022] DELETE /identity/v3/users/9cda7278efbe4e37af96ca1e32888a06 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:52.000786 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfdd2c-32be-4cf5-b96d-25764279096f None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:52.001252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfdd2c-32be-4cf5-b96d-25764279096f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:52.001493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfdd2c-32be-4cf5-b96d-25764279096f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:52.020223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-cbcfdd2c-32be-4cf5-b96d-25764279096f None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:52.021409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cbcfdd2c-32be-4cf5-b96d-25764279096f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:52.076908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 915/1818] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:52.087020 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:52.088596 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:52.149385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a0498-774e-4c06-8ca9-7a83f3b9ef89 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:52.152845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b03e1-924e-4267-b635-00352cd14e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:52.153491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b03e1-924e-4267-b635-00352cd14e2e service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:52.153772 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b03e1-924e-4267-b635-00352cd14e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:52.154045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b03e1-924e-4267-b635-00352cd14e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:52.163383 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b03e1-924e-4267-b635-00352cd14e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:52.164884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b03e1-924e-4267-b635-00352cd14e2e service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:52.174963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 904/1819] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:52.443937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfdd2c-32be-4cf5-b96d-25764279096f None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:52.446146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cbcfdd2c-32be-4cf5-b96d-25764279096f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:52.463541 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfdd2c-32be-4cf5-b96d-25764279096f None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:52.470469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:52.471348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:52.471701 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:52.472129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] PATH_INFO: `/v3/projects/776ca5a21dd2425a95ebd514efbabee7` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:52.473556 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=776ca5a21dd2425a95ebd514efbabee7)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:52.475474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:52.521540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '776ca5a21dd2425a95ebd514efbabee7', 'request_id': 'req-2c843529-1803-4a93-913a-43c2e9474144'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:52.526664 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '776ca5a21dd2425a95ebd514efbabee7', 'request_id': 'req-2c843529-1803-4a93-913a-43c2e9474144'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:52.533916 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '776ca5a21dd2425a95ebd514efbabee7', 'request_id': 'req-2c843529-1803-4a93-913a-43c2e9474144'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:52.544902 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] The token cache is being invalidate because project 776ca5a21dd2425a95ebd514efbabee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:52.545184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:52.545904 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:52.547467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 916/1820] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:52 2022] DELETE /identity/v3/projects/776ca5a21dd2425a95ebd514efbabee7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:52.551347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b03e1-924e-4267-b635-00352cd14e2e service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:52.561116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b03e1-924e-4267-b635-00352cd14e2e service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:52.616980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2a272-97b4-4a20-b9fc-af403d01ddd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:52.617865 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a272-97b4-4a20-b9fc-af403d01ddd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:52.618147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a272-97b4-4a20-b9fc-af403d01ddd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:52.618420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a272-97b4-4a20-b9fc-af403d01ddd0 admin admin] PATH_INFO: `/v3/users/0ff9f97260e54571aa749fa07524e5fd` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:52.619655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2a272-97b4-4a20-b9fc-af403d01ddd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ff9f97260e54571aa749fa07524e5fd)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:52.621634 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2a272-97b4-4a20-b9fc-af403d01ddd0 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:52.675323 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-64f2a272-97b4-4a20-b9fc-af403d01ddd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ff9f97260e54571aa749fa07524e5fd'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:52.676030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-64f2a272-97b4-4a20-b9fc-af403d01ddd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ff9f97260e54571aa749fa07524e5fd'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:52.676660 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-64f2a272-97b4-4a20-b9fc-af403d01ddd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ff9f97260e54571aa749fa07524e5fd'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:52.685232 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-64f2a272-97b4-4a20-b9fc-af403d01ddd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ff9f97260e54571aa749fa07524e5fd'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:52.692290 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-64f2a272-97b4-4a20-b9fc-af403d01ddd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ff9f97260e54571aa749fa07524e5fd'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:52.712546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 905/1821] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:52 2022] DELETE /identity/v3/users/0ff9f97260e54571aa749fa07524e5fd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:52.996753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:52.999369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:53.013681 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c843529-1803-4a93-913a-43c2e9474144 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:53.071661 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76635147-be11-48d6-a46e-c1c6e732ccb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:53.072789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76635147-be11-48d6-a46e-c1c6e732ccb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.073073 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76635147-be11-48d6-a46e-c1c6e732ccb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.073335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76635147-be11-48d6-a46e-c1c6e732ccb6 admin admin] PATH_INFO: `/v3/projects/c8591337c62d4ea29d08891e0e016ce6` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.074514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76635147-be11-48d6-a46e-c1c6e732ccb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8591337c62d4ea29d08891e0e016ce6)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:53.076474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76635147-be11-48d6-a46e-c1c6e732ccb6 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:53.123402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-76635147-be11-48d6-a46e-c1c6e732ccb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8591337c62d4ea29d08891e0e016ce6', 'request_id': 'req-76635147-be11-48d6-a46e-c1c6e732ccb6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:53.128845 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-76635147-be11-48d6-a46e-c1c6e732ccb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8591337c62d4ea29d08891e0e016ce6', 'request_id': 'req-76635147-be11-48d6-a46e-c1c6e732ccb6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:53.136124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-76635147-be11-48d6-a46e-c1c6e732ccb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8591337c62d4ea29d08891e0e016ce6', 'request_id': 'req-76635147-be11-48d6-a46e-c1c6e732ccb6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:53.147386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-76635147-be11-48d6-a46e-c1c6e732ccb6 admin admin] The token cache is being invalidate because project c8591337c62d4ea29d08891e0e016ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:53.147721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-76635147-be11-48d6-a46e-c1c6e732ccb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:53.148902 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-76635147-be11-48d6-a46e-c1c6e732ccb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:53.151161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 917/1822] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:52 2022] DELETE /identity/v3/projects/c8591337c62d4ea29d08891e0e016ce6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:53.156846 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d8fb3-0a19-4eac-9e05-7fc7c96dc14f None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.157319 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d8fb3-0a19-4eac-9e05-7fc7c96dc14f None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.157420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d8fb3-0a19-4eac-9e05-7fc7c96dc14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.177579 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-fd4d8fb3-0a19-4eac-9e05-7fc7c96dc14f None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:53.178771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-fd4d8fb3-0a19-4eac-9e05-7fc7c96dc14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:53.233646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 906/1823] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:53.243615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f9f5f-f57c-40b4-ab10-1d7736e8801c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.243937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f9f5f-f57c-40b4-ab10-1d7736e8801c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.244248 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f9f5f-f57c-40b4-ab10-1d7736e8801c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.261840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-004f9f5f-f57c-40b4-ab10-1d7736e8801c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:53.263073 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-004f9f5f-f57c-40b4-ab10-1d7736e8801c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:53.270830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 918/1824] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:53.275554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d8fb3-0a19-4eac-9e05-7fc7c96dc14f None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:53.284913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d8fb3-0a19-4eac-9e05-7fc7c96dc14f None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:53.289953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a895f9-d084-443e-99de-f2169b3743de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:53.290599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a895f9-d084-443e-99de-f2169b3743de admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.290878 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a895f9-d084-443e-99de-f2169b3743de admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.291148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a895f9-d084-443e-99de-f2169b3743de admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.291862 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a895f9-d084-443e-99de-f2169b3743de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:53.292194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a895f9-d084-443e-99de-f2169b3743de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:53.293761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a895f9-d084-443e-99de-f2169b3743de admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:53.305548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 907/1825] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:26:53.310117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f9f5f-f57c-40b4-ab10-1d7736e8801c None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:53.319417 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f9f5f-f57c-40b4-ab10-1d7736e8801c None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:53.324079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd3dd6-fc38-46c9-840c-61ec16681ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:53.324729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd3dd6-fc38-46c9-840c-61ec16681ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.325008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd3dd6-fc38-46c9-840c-61ec16681ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.325280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd3dd6-fc38-46c9-840c-61ec16681ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.325976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd3dd6-fc38-46c9-840c-61ec16681ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:53.327839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd3dd6-fc38-46c9-840c-61ec16681ec9 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:53.354156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 919/1826] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:53 2022] POST /identity/v3/projects => generated 401 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:53.358228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a895f9-d084-443e-99de-f2169b3743de admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:53.369029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a895f9-d084-443e-99de-f2169b3743de admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:53.424159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bf0ff-3c91-4aa0-a658-fe6080489cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:53.424825 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bf0ff-3c91-4aa0-a658-fe6080489cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.425112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bf0ff-3c91-4aa0-a658-fe6080489cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.425391 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bf0ff-3c91-4aa0-a658-fe6080489cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.426083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6bf0ff-3c91-4aa0-a658-fe6080489cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:53.427997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6bf0ff-3c91-4aa0-a658-fe6080489cd0 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:53.447150 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 908/1827] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:53 2022] POST /identity/v3/users => generated 301 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:26:53.451195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd3dd6-fc38-46c9-840c-61ec16681ec9 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:53.461834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd3dd6-fc38-46c9-840c-61ec16681ec9 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:53.466437 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507a41e-31d6-4ac1-a148-23fcfab449d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:53.467127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507a41e-31d6-4ac1-a148-23fcfab449d3 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.467403 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507a41e-31d6-4ac1-a148-23fcfab449d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.467674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507a41e-31d6-4ac1-a148-23fcfab449d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.468433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507a41e-31d6-4ac1-a148-23fcfab449d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:53.469999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507a41e-31d6-4ac1-a148-23fcfab449d3 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:53.483496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 920/1828] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:53.484637 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bf0ff-3c91-4aa0-a658-fe6080489cd0 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:53.494298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bf0ff-3c91-4aa0-a658-fe6080489cd0 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:53.498800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53119e-eb55-426f-852c-fcdc60ba856d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:53.499448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53119e-eb55-426f-852c-fcdc60ba856d admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.499728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53119e-eb55-426f-852c-fcdc60ba856d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.500036 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53119e-eb55-426f-852c-fcdc60ba856d admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.500754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53119e-eb55-426f-852c-fcdc60ba856d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:53.502301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53119e-eb55-426f-852c-fcdc60ba856d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:53.511808 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 909/1829] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:53.515888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507a41e-31d6-4ac1-a148-23fcfab449d3 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:53.526201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507a41e-31d6-4ac1-a148-23fcfab449d3 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:53.530694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba659f-080b-4315-8206-40dac9b0bfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:53.531597 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba659f-080b-4315-8206-40dac9b0bfba admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.531879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba659f-080b-4315-8206-40dac9b0bfba admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.532180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba659f-080b-4315-8206-40dac9b0bfba admin admin] PATH_INFO: `/v3/projects/02a82982b3784294a5222c76163e356f/users/70d00a47e1e8498fbad0de2955283f23/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.546988 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba659f-080b-4315-8206-40dac9b0bfba admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a82982b3784294a5222c76163e356f, user_id=70d00a47e1e8498fbad0de2955283f23, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:53.549781 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba659f-080b-4315-8206-40dac9b0bfba admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:53.563736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 921/1830] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:53 2022] PUT /identity/v3/projects/02a82982b3784294a5222c76163e356f/users/70d00a47e1e8498fbad0de2955283f23/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:53.574605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082af75a-7c53-4a55-9cb2-55e8a592eea4 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.574842 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082af75a-7c53-4a55-9cb2-55e8a592eea4 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.575127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082af75a-7c53-4a55-9cb2-55e8a592eea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.601356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-082af75a-7c53-4a55-9cb2-55e8a592eea4 None None] MFA Rules not processed for user `70d00a47e1e8498fbad0de2955283f23`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:53.602541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-082af75a-7c53-4a55-9cb2-55e8a592eea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:53.662763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 910/1831] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:53.667325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba659f-080b-4315-8206-40dac9b0bfba admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:53.728815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba659f-080b-4315-8206-40dac9b0bfba admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:53.734271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b09d1c-c2e6-43f2-9a3f-6e806908ada5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:53.734996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b09d1c-c2e6-43f2-9a3f-6e806908ada5 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.735325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b09d1c-c2e6-43f2-9a3f-6e806908ada5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.735482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b09d1c-c2e6-43f2-9a3f-6e806908ada5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.736190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b09d1c-c2e6-43f2-9a3f-6e806908ada5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:53.738074 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b09d1c-c2e6-43f2-9a3f-6e806908ada5 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:53.778268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 922/1832] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:53 2022] POST /identity/v3/projects => generated 401 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:53.782458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082af75a-7c53-4a55-9cb2-55e8a592eea4 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:53.795937 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082af75a-7c53-4a55-9cb2-55e8a592eea4 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:53.858553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345ceed-5448-4498-964a-d79844c1431d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:53.859242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345ceed-5448-4498-964a-d79844c1431d admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.859523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345ceed-5448-4498-964a-d79844c1431d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.859802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345ceed-5448-4498-964a-d79844c1431d admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.860528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345ceed-5448-4498-964a-d79844c1431d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:53.862419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345ceed-5448-4498-964a-d79844c1431d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:53.884819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 911/1833] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:53 2022] POST /identity/v3/users => generated 307 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:26:53.888931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b09d1c-c2e6-43f2-9a3f-6e806908ada5 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:53.899334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b09d1c-c2e6-43f2-9a3f-6e806908ada5 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:53.904068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383c8ee-627e-4dbc-a4bd-5ffe86cbf764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:53.904752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383c8ee-627e-4dbc-a4bd-5ffe86cbf764 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.905066 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383c8ee-627e-4dbc-a4bd-5ffe86cbf764 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.905313 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383c8ee-627e-4dbc-a4bd-5ffe86cbf764 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.906188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383c8ee-627e-4dbc-a4bd-5ffe86cbf764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:53.907811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383c8ee-627e-4dbc-a4bd-5ffe86cbf764 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:53.919453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 923/1834] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:53.923454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345ceed-5448-4498-964a-d79844c1431d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:53.934012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345ceed-5448-4498-964a-d79844c1431d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:53.939532 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971de32-3142-408f-98fe-4fe579ef2df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:53.940455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971de32-3142-408f-98fe-4fe579ef2df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:53.940736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971de32-3142-408f-98fe-4fe579ef2df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:53.941146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971de32-3142-408f-98fe-4fe579ef2df0 admin admin] PATH_INFO: `/v3/projects/966b6209b73c4d6989ab2f2f469a3b35/users/84082b7a602c4de29200b62d8a2314a3/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:53.956170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971de32-3142-408f-98fe-4fe579ef2df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966b6209b73c4d6989ab2f2f469a3b35, user_id=84082b7a602c4de29200b62d8a2314a3, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:53.958846 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971de32-3142-408f-98fe-4fe579ef2df0 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:53.972063 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 912/1835] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:53 2022] PUT /identity/v3/projects/966b6209b73c4d6989ab2f2f469a3b35/users/84082b7a602c4de29200b62d8a2314a3/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:53.976622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383c8ee-627e-4dbc-a4bd-5ffe86cbf764 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:53.986329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383c8ee-627e-4dbc-a4bd-5ffe86cbf764 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:54.042422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2d03a-b029-4d38-9301-d9900ad73f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:54.043216 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2d03a-b029-4d38-9301-d9900ad73f9d admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:54.043474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2d03a-b029-4d38-9301-d9900ad73f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:54.043752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2d03a-b029-4d38-9301-d9900ad73f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:54.044600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2d03a-b029-4d38-9301-d9900ad73f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:54.046413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2d03a-b029-4d38-9301-d9900ad73f9d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:54.059295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 924/1836] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:54.063266 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971de32-3142-408f-98fe-4fe579ef2df0 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:54.074434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971de32-3142-408f-98fe-4fe579ef2df0 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:54.079045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d135b36-3138-4749-b9d2-3607eca1e006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:54.079974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d135b36-3138-4749-b9d2-3607eca1e006 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:54.080387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d135b36-3138-4749-b9d2-3607eca1e006 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:54.080662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d135b36-3138-4749-b9d2-3607eca1e006 admin admin] PATH_INFO: `/v3/projects/966b6209b73c4d6989ab2f2f469a3b35/users/84082b7a602c4de29200b62d8a2314a3/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:54.082645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d135b36-3138-4749-b9d2-3607eca1e006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966b6209b73c4d6989ab2f2f469a3b35, user_id=84082b7a602c4de29200b62d8a2314a3, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:54.085441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d135b36-3138-4749-b9d2-3607eca1e006 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:54.095501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 913/1837] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:54 2022] PUT /identity/v3/projects/966b6209b73c4d6989ab2f2f469a3b35/users/84082b7a602c4de29200b62d8a2314a3/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:26:54.107012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30802101-7d5f-4182-b84e-dc5b58985a92 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:54.107286 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30802101-7d5f-4182-b84e-dc5b58985a92 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:54.107786 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30802101-7d5f-4182-b84e-dc5b58985a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:54.137202 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-30802101-7d5f-4182-b84e-dc5b58985a92 None None] MFA Rules not processed for user `84082b7a602c4de29200b62d8a2314a3`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:54.138443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-30802101-7d5f-4182-b84e-dc5b58985a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:54.211028 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 925/1838] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:54 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:54.222737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d135b36-3138-4749-b9d2-3607eca1e006 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:54.226590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-7d135b36-3138-4749-b9d2-3607eca1e006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:54.290910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d135b36-3138-4749-b9d2-3607eca1e006 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:54.295169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c1a2b-441b-4d8b-b0f1-2017a1972522 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:54.296255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1a2b-441b-4d8b-b0f1-2017a1972522 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:54.296535 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1a2b-441b-4d8b-b0f1-2017a1972522 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:54.296910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1a2b-441b-4d8b-b0f1-2017a1972522 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:54.307822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c1a2b-441b-4d8b-b0f1-2017a1972522 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:54.309293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c1a2b-441b-4d8b-b0f1-2017a1972522 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:54.325603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 914/1839] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:54 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:54.401738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30802101-7d5f-4182-b84e-dc5b58985a92 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:54.411167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30802101-7d5f-4182-b84e-dc5b58985a92 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:54.415625 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e9526-2064-4dfa-9312-c3072ea82d68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:54.416409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e9526-2064-4dfa-9312-c3072ea82d68 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:54.416670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e9526-2064-4dfa-9312-c3072ea82d68 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:54.416949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e9526-2064-4dfa-9312-c3072ea82d68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:54.474575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e9526-2064-4dfa-9312-c3072ea82d68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:54.476054 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e9526-2064-4dfa-9312-c3072ea82d68 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:54.485115 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 926/1840] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:54.661055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c1a2b-441b-4d8b-b0f1-2017a1972522 service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:54.663854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-ac6c1a2b-441b-4d8b-b0f1-2017a1972522 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:54.678131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c1a2b-441b-4d8b-b0f1-2017a1972522 service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:54.683080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7af971-b032-4ef8-8d29-f208f7e2106f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:54.683966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af971-b032-4ef8-8d29-f208f7e2106f service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:54.684467 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af971-b032-4ef8-8d29-f208f7e2106f service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:54.684848 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af971-b032-4ef8-8d29-f208f7e2106f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:54.687124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-ce7af971-b032-4ef8-8d29-f208f7e2106f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:54.812398 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7af971-b032-4ef8-8d29-f208f7e2106f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:54.815173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7af971-b032-4ef8-8d29-f208f7e2106f service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:54.845658 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 915/1841] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:55.046298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e9526-2064-4dfa-9312-c3072ea82d68 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:55.048830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4f5e9526-2064-4dfa-9312-c3072ea82d68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:55.221972 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e9526-2064-4dfa-9312-c3072ea82d68 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:55.234183 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:55.235075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:55.235353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:55.235622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] PATH_INFO: `/v3/users/ea5874eef71e43b7abed80569cd766ac` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:55.248854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea5874eef71e43b7abed80569cd766ac)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:55.250872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:55.340532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea5874eef71e43b7abed80569cd766ac'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:55.342540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea5874eef71e43b7abed80569cd766ac'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:55.353414 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea5874eef71e43b7abed80569cd766ac'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:55.363194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea5874eef71e43b7abed80569cd766ac'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:55.364780 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea5874eef71e43b7abed80569cd766ac'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:55.389725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 927/1842] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:55 2022] DELETE /identity/v3/users/ea5874eef71e43b7abed80569cd766ac => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:55.394016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7525c70-4d51-46c4-8342-5b2b04a754bd None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:55.394839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7525c70-4d51-46c4-8342-5b2b04a754bd None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:55.395270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7525c70-4d51-46c4-8342-5b2b04a754bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:55.428139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-b7525c70-4d51-46c4-8342-5b2b04a754bd None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:55.429628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b7525c70-4d51-46c4-8342-5b2b04a754bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:55.513040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 916/1843] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:55.562516 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:55.564578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:55.651937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58c613-c131-4d6b-91b5-7a599e742690 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:55.658747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea50010-8746-4a85-ab98-e5fadba28707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:55.658747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea50010-8746-4a85-ab98-e5fadba28707 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:55.658747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea50010-8746-4a85-ab98-e5fadba28707 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:55.658747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea50010-8746-4a85-ab98-e5fadba28707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:55.682298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea50010-8746-4a85-ab98-e5fadba28707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:55.684220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea50010-8746-4a85-ab98-e5fadba28707 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:55.698179 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 928/1844] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:56.304138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7525c70-4d51-46c4-8342-5b2b04a754bd None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:56.305765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b7525c70-4d51-46c4-8342-5b2b04a754bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:56.326798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7525c70-4d51-46c4-8342-5b2b04a754bd None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:56.340689 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:56.342037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:56.342577 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:56.342990 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] PATH_INFO: `/v3/projects/0dd9e10217e64a2a9bd5673a803ac71f` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:56.345682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dd9e10217e64a2a9bd5673a803ac71f)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:56.349803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:56.418811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dd9e10217e64a2a9bd5673a803ac71f', 'request_id': 'req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:56.426406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dd9e10217e64a2a9bd5673a803ac71f', 'request_id': 'req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:56.434440 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dd9e10217e64a2a9bd5673a803ac71f', 'request_id': 'req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:56.449648 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] The token cache is being invalidate because project 0dd9e10217e64a2a9bd5673a803ac71f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:56.449951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:56.450602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:56.452401 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 917/1845] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:56 2022] DELETE /identity/v3/projects/0dd9e10217e64a2a9bd5673a803ac71f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:56.456443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea50010-8746-4a85-ab98-e5fadba28707 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:56.466690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea50010-8746-4a85-ab98-e5fadba28707 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:56.531774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe201072-2ae2-49c8-ac88-eded198ea3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:56.532721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe201072-2ae2-49c8-ac88-eded198ea3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:56.533001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe201072-2ae2-49c8-ac88-eded198ea3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:56.533282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe201072-2ae2-49c8-ac88-eded198ea3f8 admin admin] PATH_INFO: `/v3/users/e6bd465b458146f48bd290ffe35743ee` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:56.534654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe201072-2ae2-49c8-ac88-eded198ea3f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6bd465b458146f48bd290ffe35743ee)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:56.536911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe201072-2ae2-49c8-ac88-eded198ea3f8 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:56.603357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fe201072-2ae2-49c8-ac88-eded198ea3f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6bd465b458146f48bd290ffe35743ee'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:56.604065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fe201072-2ae2-49c8-ac88-eded198ea3f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6bd465b458146f48bd290ffe35743ee'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:56.612054 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fe201072-2ae2-49c8-ac88-eded198ea3f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6bd465b458146f48bd290ffe35743ee'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:56.619612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fe201072-2ae2-49c8-ac88-eded198ea3f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6bd465b458146f48bd290ffe35743ee'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:56.620290 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fe201072-2ae2-49c8-ac88-eded198ea3f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6bd465b458146f48bd290ffe35743ee'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:56.638242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 929/1846] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:56 2022] DELETE /identity/v3/users/e6bd465b458146f48bd290ffe35743ee => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:57.005055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:57.006699 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:57.020940 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6d58c-12e9-4ca5-92e3-5a8f5a89e10d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:57.077552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a882b-67e9-46ca-9c99-250f066a6a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:57.078415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a882b-67e9-46ca-9c99-250f066a6a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:57.078703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a882b-67e9-46ca-9c99-250f066a6a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:57.078979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a882b-67e9-46ca-9c99-250f066a6a77 admin admin] PATH_INFO: `/v3/projects/325adb362e0545a8ae52bdc4a6faea3e` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:57.080178 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a882b-67e9-46ca-9c99-250f066a6a77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=325adb362e0545a8ae52bdc4a6faea3e)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:57.082054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a882b-67e9-46ca-9c99-250f066a6a77 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:57.125806 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1b9a882b-67e9-46ca-9c99-250f066a6a77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '325adb362e0545a8ae52bdc4a6faea3e', 'request_id': 'req-1b9a882b-67e9-46ca-9c99-250f066a6a77'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:57.131683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1b9a882b-67e9-46ca-9c99-250f066a6a77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '325adb362e0545a8ae52bdc4a6faea3e', 'request_id': 'req-1b9a882b-67e9-46ca-9c99-250f066a6a77'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:57.136848 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1b9a882b-67e9-46ca-9c99-250f066a6a77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '325adb362e0545a8ae52bdc4a6faea3e', 'request_id': 'req-1b9a882b-67e9-46ca-9c99-250f066a6a77'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:57.149713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1b9a882b-67e9-46ca-9c99-250f066a6a77 admin admin] The token cache is being invalidate because project 325adb362e0545a8ae52bdc4a6faea3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:26:57.150000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1b9a882b-67e9-46ca-9c99-250f066a6a77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:57.150618 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1b9a882b-67e9-46ca-9c99-250f066a6a77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:57.152285 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 918/1847] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:26:57 2022] DELETE /identity/v3/projects/325adb362e0545a8ae52bdc4a6faea3e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:57.157172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afb49c-7bcc-4355-8fbe-e6ec5c43ea79 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:57.157467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afb49c-7bcc-4355-8fbe-e6ec5c43ea79 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:57.157752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afb49c-7bcc-4355-8fbe-e6ec5c43ea79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:57.175763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-40afb49c-7bcc-4355-8fbe-e6ec5c43ea79 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:57.177389 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-40afb49c-7bcc-4355-8fbe-e6ec5c43ea79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:57.233219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 930/1848] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:57.240887 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6989340-a7b7-41bd-ada4-1a720b41df28 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:57.241176 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6989340-a7b7-41bd-ada4-1a720b41df28 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:57.241453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6989340-a7b7-41bd-ada4-1a720b41df28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:57.258820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-a6989340-a7b7-41bd-ada4-1a720b41df28 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:57.260136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a6989340-a7b7-41bd-ada4-1a720b41df28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:57.267722 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 919/1849] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:26:57.272667 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afb49c-7bcc-4355-8fbe-e6ec5c43ea79 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:57.282135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afb49c-7bcc-4355-8fbe-e6ec5c43ea79 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:57.287250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a6e5b-af2a-474b-a51f-8367a807ea16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:57.287911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a6e5b-af2a-474b-a51f-8367a807ea16 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:57.288205 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a6e5b-af2a-474b-a51f-8367a807ea16 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:57.288472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a6e5b-af2a-474b-a51f-8367a807ea16 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:57.289188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a6e5b-af2a-474b-a51f-8367a807ea16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:26:57.289488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a6e5b-af2a-474b-a51f-8367a807ea16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:57.291062 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a6e5b-af2a-474b-a51f-8367a807ea16 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:57.302779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 931/1850] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:26:57.307183 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6989340-a7b7-41bd-ada4-1a720b41df28 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:57.316456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6989340-a7b7-41bd-ada4-1a720b41df28 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:57.321426 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9fcf9-1688-4ca7-8e89-c3df701254a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:57.322088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9fcf9-1688-4ca7-8e89-c3df701254a7 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:57.322373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9fcf9-1688-4ca7-8e89-c3df701254a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:57.322651 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9fcf9-1688-4ca7-8e89-c3df701254a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:57.323337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9fcf9-1688-4ca7-8e89-c3df701254a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:57.325279 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9fcf9-1688-4ca7-8e89-c3df701254a7 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:57.350479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 920/1851] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:57 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:57.354471 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a6e5b-af2a-474b-a51f-8367a807ea16 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:57.364108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a6e5b-af2a-474b-a51f-8367a807ea16 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:57.422058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eabf7-917c-4002-818d-874c250cef8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:57.422734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eabf7-917c-4002-818d-874c250cef8e admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:57.423012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eabf7-917c-4002-818d-874c250cef8e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:57.423479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eabf7-917c-4002-818d-874c250cef8e admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:57.424173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eabf7-917c-4002-818d-874c250cef8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:57.426057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eabf7-917c-4002-818d-874c250cef8e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:57.452579 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 932/1852] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:57 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:26:57.456522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9fcf9-1688-4ca7-8e89-c3df701254a7 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:57.468323 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9fcf9-1688-4ca7-8e89-c3df701254a7 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:57.474228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09c6d1-80e7-4863-bb41-30d240c17d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:57.475471 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09c6d1-80e7-4863-bb41-30d240c17d71 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:57.475756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09c6d1-80e7-4863-bb41-30d240c17d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:57.476059 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09c6d1-80e7-4863-bb41-30d240c17d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:57.476786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09c6d1-80e7-4863-bb41-30d240c17d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:57.478367 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09c6d1-80e7-4863-bb41-30d240c17d71 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:57.493690 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 921/1853] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:26:57.494767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eabf7-917c-4002-818d-874c250cef8e admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:57.504670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eabf7-917c-4002-818d-874c250cef8e admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:57.510908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dae02-c8fb-46ca-908c-ac6def560ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:57.511604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dae02-c8fb-46ca-908c-ac6def560ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:57.511919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dae02-c8fb-46ca-908c-ac6def560ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:57.512159 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dae02-c8fb-46ca-908c-ac6def560ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:57.512886 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dae02-c8fb-46ca-908c-ac6def560ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:57.515436 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dae02-c8fb-46ca-908c-ac6def560ad3 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:57.526813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 933/1854] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:26:57.530970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09c6d1-80e7-4863-bb41-30d240c17d71 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:57.543569 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09c6d1-80e7-4863-bb41-30d240c17d71 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:57.548594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dae26f-cd41-4d6b-916d-2edbcd8bbc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:57.549494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dae26f-cd41-4d6b-916d-2edbcd8bbc45 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:57.549802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dae26f-cd41-4d6b-916d-2edbcd8bbc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:57.550233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dae26f-cd41-4d6b-916d-2edbcd8bbc45 admin admin] PATH_INFO: `/v3/projects/efd18d14203d49fd816186022e30e604/users/968cae1bf670412e8cb22c5b2c0de987/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:57.574240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dae26f-cd41-4d6b-916d-2edbcd8bbc45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efd18d14203d49fd816186022e30e604, user_id=968cae1bf670412e8cb22c5b2c0de987, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:57.577412 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dae26f-cd41-4d6b-916d-2edbcd8bbc45 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:57.598439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 922/1855] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:57 2022] PUT /identity/v3/projects/efd18d14203d49fd816186022e30e604/users/968cae1bf670412e8cb22c5b2c0de987/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:57.610582 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc1f41-a118-41b1-ae5e-126c2b4cc272 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:57.611051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc1f41-a118-41b1-ae5e-126c2b4cc272 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:57.611328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc1f41-a118-41b1-ae5e-126c2b4cc272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:57.639658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-6afc1f41-a118-41b1-ae5e-126c2b4cc272 None None] MFA Rules not processed for user `968cae1bf670412e8cb22c5b2c0de987`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:57.640898 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6afc1f41-a118-41b1-ae5e-126c2b4cc272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:57.720130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 934/1856] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:57.725295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dae26f-cd41-4d6b-916d-2edbcd8bbc45 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:57.794676 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dae26f-cd41-4d6b-916d-2edbcd8bbc45 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:57.800072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879ea14-4c20-4131-8bd0-86ca58d9c9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:57.800820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879ea14-4c20-4131-8bd0-86ca58d9c9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:57.801174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879ea14-4c20-4131-8bd0-86ca58d9c9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:57.801520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879ea14-4c20-4131-8bd0-86ca58d9c9b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:57.802366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879ea14-4c20-4131-8bd0-86ca58d9c9b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:57.804506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879ea14-4c20-4131-8bd0-86ca58d9c9b8 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:57.831130 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 923/1857] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:26:57 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:26:57.836273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc1f41-a118-41b1-ae5e-126c2b4cc272 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:57.853570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc1f41-a118-41b1-ae5e-126c2b4cc272 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:57.907314 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194343f-d5de-4a02-a953-5a5230b4b95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:57.908027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194343f-d5de-4a02-a953-5a5230b4b95d admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:57.908314 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194343f-d5de-4a02-a953-5a5230b4b95d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:57.908887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194343f-d5de-4a02-a953-5a5230b4b95d admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:57.909280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194343f-d5de-4a02-a953-5a5230b4b95d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:57.911229 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194343f-d5de-4a02-a953-5a5230b4b95d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:57.934423 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 935/1858] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:26:57 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:26:57.935590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879ea14-4c20-4131-8bd0-86ca58d9c9b8 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:57.946173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879ea14-4c20-4131-8bd0-86ca58d9c9b8 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:57.965539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbb93d-1018-464e-b152-2333b2a26a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:57.966336 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbb93d-1018-464e-b152-2333b2a26a2b admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:57.966694 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbb93d-1018-464e-b152-2333b2a26a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:57.967046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbb93d-1018-464e-b152-2333b2a26a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:57.968059 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbb93d-1018-464e-b152-2333b2a26a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:57.969735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbb93d-1018-464e-b152-2333b2a26a2b admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:57.985991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 924/1859] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:57.987466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194343f-d5de-4a02-a953-5a5230b4b95d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:57.997983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194343f-d5de-4a02-a953-5a5230b4b95d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:58.002743 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09169fd1-12fa-46fc-a115-97dd1f5e006d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:58.003642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09169fd1-12fa-46fc-a115-97dd1f5e006d admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:58.004004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09169fd1-12fa-46fc-a115-97dd1f5e006d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:58.004230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09169fd1-12fa-46fc-a115-97dd1f5e006d admin admin] PATH_INFO: `/v3/projects/bdf81c906ff94ef08d709ec14305418f/users/251cc65a865046c8a0770d7169f98f5c/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:58.021593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09169fd1-12fa-46fc-a115-97dd1f5e006d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf81c906ff94ef08d709ec14305418f, user_id=251cc65a865046c8a0770d7169f98f5c, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:58.024058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09169fd1-12fa-46fc-a115-97dd1f5e006d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:58.038414 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 936/1860] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:57 2022] PUT /identity/v3/projects/bdf81c906ff94ef08d709ec14305418f/users/251cc65a865046c8a0770d7169f98f5c/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:58.042268 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbb93d-1018-464e-b152-2333b2a26a2b admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:58.052363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbb93d-1018-464e-b152-2333b2a26a2b admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:58.133275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5f615-7348-4966-8f5b-8a348fea8d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:58.133959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5f615-7348-4966-8f5b-8a348fea8d06 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:58.134240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5f615-7348-4966-8f5b-8a348fea8d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:58.134518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5f615-7348-4966-8f5b-8a348fea8d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:58.135247 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5f615-7348-4966-8f5b-8a348fea8d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:58.136902 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5f615-7348-4966-8f5b-8a348fea8d06 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:58.147086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 925/1861] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:26:58.151250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09169fd1-12fa-46fc-a115-97dd1f5e006d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:58.161876 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09169fd1-12fa-46fc-a115-97dd1f5e006d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:58.166544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9a1ce-0273-4b81-b9cd-5116c7e86cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:58.167425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9a1ce-0273-4b81-b9cd-5116c7e86cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:58.167709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9a1ce-0273-4b81-b9cd-5116c7e86cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:58.168012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9a1ce-0273-4b81-b9cd-5116c7e86cf7 admin admin] PATH_INFO: `/v3/projects/bdf81c906ff94ef08d709ec14305418f/users/251cc65a865046c8a0770d7169f98f5c/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:58.169971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9a1ce-0273-4b81-b9cd-5116c7e86cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf81c906ff94ef08d709ec14305418f, user_id=251cc65a865046c8a0770d7169f98f5c, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:58.172642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9a1ce-0273-4b81-b9cd-5116c7e86cf7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:58.182848 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 937/1862] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:26:58 2022] PUT /identity/v3/projects/bdf81c906ff94ef08d709ec14305418f/users/251cc65a865046c8a0770d7169f98f5c/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:58.193169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f1017-80b0-457c-a48c-2eb0277d0779 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:58.193468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f1017-80b0-457c-a48c-2eb0277d0779 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:58.193779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f1017-80b0-457c-a48c-2eb0277d0779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:58.220551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-ef9f1017-80b0-457c-a48c-2eb0277d0779 None None] MFA Rules not processed for user `251cc65a865046c8a0770d7169f98f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:58.221748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-ef9f1017-80b0-457c-a48c-2eb0277d0779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:58.287919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 926/1863] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:26:58.299119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9a1ce-0273-4b81-b9cd-5116c7e86cf7 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:58.300844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-eda9a1ce-0273-4b81-b9cd-5116c7e86cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:58.356481 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9a1ce-0273-4b81-b9cd-5116c7e86cf7 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:58.360070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068ae97-29d1-444a-82ac-ae3bc8726aec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:58.360686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068ae97-29d1-444a-82ac-ae3bc8726aec service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:58.360961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068ae97-29d1-444a-82ac-ae3bc8726aec service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:58.361230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068ae97-29d1-444a-82ac-ae3bc8726aec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:58.370428 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068ae97-29d1-444a-82ac-ae3bc8726aec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:58.371915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2068ae97-29d1-444a-82ac-ae3bc8726aec service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:58.381641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 938/1864] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:58.464413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f1017-80b0-457c-a48c-2eb0277d0779 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:58.473594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f1017-80b0-457c-a48c-2eb0277d0779 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:58.477218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921837d-e679-4860-9809-c10b4edb3b76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:58.477857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921837d-e679-4860-9809-c10b4edb3b76 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:58.478135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921837d-e679-4860-9809-c10b4edb3b76 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:58.478476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921837d-e679-4860-9809-c10b4edb3b76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:58.565974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921837d-e679-4860-9809-c10b4edb3b76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:58.568320 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921837d-e679-4860-9809-c10b4edb3b76 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:58.584720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 927/1865] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:26:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:26:59.776032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068ae97-29d1-444a-82ac-ae3bc8726aec service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:26:59.778199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2068ae97-29d1-444a-82ac-ae3bc8726aec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:26:59.861344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2068ae97-29d1-444a-82ac-ae3bc8726aec service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:26:59.867692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:26:59.868615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:59.868905 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:59.869188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] PATH_INFO: `/v3/users/70d00a47e1e8498fbad0de2955283f23` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:59.870437 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d00a47e1e8498fbad0de2955283f23)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:26:59.872559 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:26:59.931980 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d00a47e1e8498fbad0de2955283f23'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:59.932940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d00a47e1e8498fbad0de2955283f23'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:59.940690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d00a47e1e8498fbad0de2955283f23'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:59.948502 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d00a47e1e8498fbad0de2955283f23'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:59.949157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d00a47e1e8498fbad0de2955283f23'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:26:59.967747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 939/1866] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:26:59 2022] DELETE /identity/v3/users/70d00a47e1e8498fbad0de2955283f23 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:26:59.972675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cecfa5-bb75-46c1-aab6-8b5fb4952cb7 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:26:59.973235 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cecfa5-bb75-46c1-aab6-8b5fb4952cb7 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:26:59.973506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cecfa5-bb75-46c1-aab6-8b5fb4952cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:26:59.991055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-a2cecfa5-bb75-46c1-aab6-8b5fb4952cb7 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:26:59.992500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a2cecfa5-bb75-46c1-aab6-8b5fb4952cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:00.059513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 928/1867] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:00.067939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:00.069267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:00.138124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66dd7c-5ac1-4e96-bc29-97ebdecb1fcd admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:00.141858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63752e5d-f7ae-490c-b28f-c3bb707c10c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:00.142516 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63752e5d-f7ae-490c-b28f-c3bb707c10c6 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:00.142794 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63752e5d-f7ae-490c-b28f-c3bb707c10c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:00.143071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63752e5d-f7ae-490c-b28f-c3bb707c10c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:00.153542 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63752e5d-f7ae-490c-b28f-c3bb707c10c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:00.155108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63752e5d-f7ae-490c-b28f-c3bb707c10c6 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:00.166193 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 940/1868] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:00.619603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cecfa5-bb75-46c1-aab6-8b5fb4952cb7 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:00.621105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a2cecfa5-bb75-46c1-aab6-8b5fb4952cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:00.649033 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cecfa5-bb75-46c1-aab6-8b5fb4952cb7 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:00.659216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:00.660068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:00.661191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:00.661800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] PATH_INFO: `/v3/projects/02a82982b3784294a5222c76163e356f` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:00.663800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02a82982b3784294a5222c76163e356f)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:00.666844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:00.739766 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02a82982b3784294a5222c76163e356f', 'request_id': 'req-0a47fc62-1a75-4739-bd28-2c340a219288'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:00.749353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02a82982b3784294a5222c76163e356f', 'request_id': 'req-0a47fc62-1a75-4739-bd28-2c340a219288'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:00.757582 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02a82982b3784294a5222c76163e356f', 'request_id': 'req-0a47fc62-1a75-4739-bd28-2c340a219288'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:00.778113 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] The token cache is being invalidate because project 02a82982b3784294a5222c76163e356f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:00.778426 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:00.779407 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:00.781638 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 929/1869] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:00 2022] DELETE /identity/v3/projects/02a82982b3784294a5222c76163e356f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:00.785354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63752e5d-f7ae-490c-b28f-c3bb707c10c6 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:00.796631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63752e5d-f7ae-490c-b28f-c3bb707c10c6 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:00.869465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f27b3-25f0-4d93-bfa0-cb976315b944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:00.869465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f27b3-25f0-4d93-bfa0-cb976315b944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:00.869757 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f27b3-25f0-4d93-bfa0-cb976315b944 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:00.869757 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f27b3-25f0-4d93-bfa0-cb976315b944 admin admin] PATH_INFO: `/v3/users/84082b7a602c4de29200b62d8a2314a3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:00.870985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f27b3-25f0-4d93-bfa0-cb976315b944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84082b7a602c4de29200b62d8a2314a3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:00.872955 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f27b3-25f0-4d93-bfa0-cb976315b944 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:00.935368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c39f27b3-25f0-4d93-bfa0-cb976315b944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84082b7a602c4de29200b62d8a2314a3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:00.936289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c39f27b3-25f0-4d93-bfa0-cb976315b944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84082b7a602c4de29200b62d8a2314a3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:00.947329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c39f27b3-25f0-4d93-bfa0-cb976315b944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84082b7a602c4de29200b62d8a2314a3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:00.956413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c39f27b3-25f0-4d93-bfa0-cb976315b944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84082b7a602c4de29200b62d8a2314a3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:00.957180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c39f27b3-25f0-4d93-bfa0-cb976315b944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84082b7a602c4de29200b62d8a2314a3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:00.976023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 941/1870] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:00 2022] DELETE /identity/v3/users/84082b7a602c4de29200b62d8a2314a3 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:01.307697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:01.309232 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:01.323874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47fc62-1a75-4739-bd28-2c340a219288 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:01.379429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddd70a-0711-454c-8bcf-ab25e390966d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:01.380789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddd70a-0711-454c-8bcf-ab25e390966d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:01.381074 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddd70a-0711-454c-8bcf-ab25e390966d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:01.381342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddd70a-0711-454c-8bcf-ab25e390966d admin admin] PATH_INFO: `/v3/projects/966b6209b73c4d6989ab2f2f469a3b35` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:01.382535 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddd70a-0711-454c-8bcf-ab25e390966d admin admin] RBAC: Authorizing `identity:delete_project(project_id=966b6209b73c4d6989ab2f2f469a3b35)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:01.384473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddd70a-0711-454c-8bcf-ab25e390966d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:01.428167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-62ddd70a-0711-454c-8bcf-ab25e390966d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '966b6209b73c4d6989ab2f2f469a3b35', 'request_id': 'req-62ddd70a-0711-454c-8bcf-ab25e390966d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:01.434006 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-62ddd70a-0711-454c-8bcf-ab25e390966d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '966b6209b73c4d6989ab2f2f469a3b35', 'request_id': 'req-62ddd70a-0711-454c-8bcf-ab25e390966d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:01.439180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-62ddd70a-0711-454c-8bcf-ab25e390966d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '966b6209b73c4d6989ab2f2f469a3b35', 'request_id': 'req-62ddd70a-0711-454c-8bcf-ab25e390966d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:01.451110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-62ddd70a-0711-454c-8bcf-ab25e390966d admin admin] The token cache is being invalidate because project 966b6209b73c4d6989ab2f2f469a3b35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:01.451398 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-62ddd70a-0711-454c-8bcf-ab25e390966d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:01.452045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-62ddd70a-0711-454c-8bcf-ab25e390966d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:01.453644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 930/1871] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:01 2022] DELETE /identity/v3/projects/966b6209b73c4d6989ab2f2f469a3b35 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:01.459111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131513b-2625-4c05-9194-3d25498ef624 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:01.459401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131513b-2625-4c05-9194-3d25498ef624 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:01.459677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131513b-2625-4c05-9194-3d25498ef624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:01.477190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-4131513b-2625-4c05-9194-3d25498ef624 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:01.478384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4131513b-2625-4c05-9194-3d25498ef624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:01.534923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 942/1872] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:01.542699 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa0f53-0de3-417b-be06-8d6fe27a9ea9 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:01.543041 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa0f53-0de3-417b-be06-8d6fe27a9ea9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:01.543325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa0f53-0de3-417b-be06-8d6fe27a9ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:01.562301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-cffa0f53-0de3-417b-be06-8d6fe27a9ea9 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:01.563493 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-cffa0f53-0de3-417b-be06-8d6fe27a9ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:01.571955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 931/1873] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:01.576021 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131513b-2625-4c05-9194-3d25498ef624 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:01.586247 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131513b-2625-4c05-9194-3d25498ef624 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:01.590720 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d46811-c39e-4810-b986-ed2fb4d2ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:01.591366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46811-c39e-4810-b986-ed2fb4d2ff8f admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:01.591646 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46811-c39e-4810-b986-ed2fb4d2ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:01.591940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46811-c39e-4810-b986-ed2fb4d2ff8f admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:01.592669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d46811-c39e-4810-b986-ed2fb4d2ff8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:01.592970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d46811-c39e-4810-b986-ed2fb4d2ff8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:01.594544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d46811-c39e-4810-b986-ed2fb4d2ff8f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:01.605525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 943/1874] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:27:01.609641 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa0f53-0de3-417b-be06-8d6fe27a9ea9 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:01.619101 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa0f53-0de3-417b-be06-8d6fe27a9ea9 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:01.623648 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa881dfa-8546-4c21-a34c-ca89178ca6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:01.624338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa881dfa-8546-4c21-a34c-ca89178ca6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:01.624617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa881dfa-8546-4c21-a34c-ca89178ca6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:01.624888 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa881dfa-8546-4c21-a34c-ca89178ca6f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:01.625605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa881dfa-8546-4c21-a34c-ca89178ca6f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:01.627575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa881dfa-8546-4c21-a34c-ca89178ca6f4 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:01.658893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 932/1875] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:01 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:27:01.660570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d46811-c39e-4810-b986-ed2fb4d2ff8f admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:01.670697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d46811-c39e-4810-b986-ed2fb4d2ff8f admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:01.731287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836b8b3-cd02-44a2-8c25-4ef303c767e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:01.732004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836b8b3-cd02-44a2-8c25-4ef303c767e2 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:01.732289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836b8b3-cd02-44a2-8c25-4ef303c767e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:01.732564 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836b8b3-cd02-44a2-8c25-4ef303c767e2 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:01.733396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836b8b3-cd02-44a2-8c25-4ef303c767e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:01.735333 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836b8b3-cd02-44a2-8c25-4ef303c767e2 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:01.763069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 944/1876] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:01 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:01.766795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa881dfa-8546-4c21-a34c-ca89178ca6f4 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:01.777687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa881dfa-8546-4c21-a34c-ca89178ca6f4 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:01.784312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c58cde-d76b-4d9d-b3be-05ed7d1fbc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:01.785050 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c58cde-d76b-4d9d-b3be-05ed7d1fbc56 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:01.785258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c58cde-d76b-4d9d-b3be-05ed7d1fbc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:01.785529 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c58cde-d76b-4d9d-b3be-05ed7d1fbc56 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:01.788363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c58cde-d76b-4d9d-b3be-05ed7d1fbc56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:01.790023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c58cde-d76b-4d9d-b3be-05ed7d1fbc56 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:01.800798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 933/1877] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:01.805230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836b8b3-cd02-44a2-8c25-4ef303c767e2 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:01.815070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836b8b3-cd02-44a2-8c25-4ef303c767e2 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:01.821682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb80cc9-64ae-4428-a0da-d7d0e96a3d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:01.822353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb80cc9-64ae-4428-a0da-d7d0e96a3d43 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:01.822640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb80cc9-64ae-4428-a0da-d7d0e96a3d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:01.822918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb80cc9-64ae-4428-a0da-d7d0e96a3d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:01.823653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb80cc9-64ae-4428-a0da-d7d0e96a3d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:01.825281 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb80cc9-64ae-4428-a0da-d7d0e96a3d43 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:01.837141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 945/1878] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:01.841306 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c58cde-d76b-4d9d-b3be-05ed7d1fbc56 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:01.853365 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c58cde-d76b-4d9d-b3be-05ed7d1fbc56 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:01.863532 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99365a-8a51-417b-8114-8e2add965d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:01.865111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99365a-8a51-417b-8114-8e2add965d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:01.865651 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99365a-8a51-417b-8114-8e2add965d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:01.865999 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99365a-8a51-417b-8114-8e2add965d9a admin admin] PATH_INFO: `/v3/projects/792260608a9e4e05aafd375134e64c79/users/84f6af5b0b8b4ae9a730f27973d91b88/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:01.882294 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99365a-8a51-417b-8114-8e2add965d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=792260608a9e4e05aafd375134e64c79, user_id=84f6af5b0b8b4ae9a730f27973d91b88, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:01.885083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99365a-8a51-417b-8114-8e2add965d9a admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:01.903030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 934/1879] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:01 2022] PUT /identity/v3/projects/792260608a9e4e05aafd375134e64c79/users/84f6af5b0b8b4ae9a730f27973d91b88/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:01.913547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd7126-c4fe-43d4-b6e7-9296ff6c3747 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:01.914019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd7126-c4fe-43d4-b6e7-9296ff6c3747 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:01.914302 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd7126-c4fe-43d4-b6e7-9296ff6c3747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:01.945264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-68bd7126-c4fe-43d4-b6e7-9296ff6c3747 None None] MFA Rules not processed for user `84f6af5b0b8b4ae9a730f27973d91b88`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:01.946832 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-68bd7126-c4fe-43d4-b6e7-9296ff6c3747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:02.029437 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 946/1880] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:02.041216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99365a-8a51-417b-8114-8e2add965d9a admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:02.042938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9a99365a-8a51-417b-8114-8e2add965d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:02.107114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99365a-8a51-417b-8114-8e2add965d9a admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:02.110932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011a371-0da2-473a-8f99-6725fa29e0d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:02.111709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011a371-0da2-473a-8f99-6725fa29e0d5 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:02.112119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011a371-0da2-473a-8f99-6725fa29e0d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:02.112497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011a371-0da2-473a-8f99-6725fa29e0d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:02.122168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011a371-0da2-473a-8f99-6725fa29e0d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:02.123742 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011a371-0da2-473a-8f99-6725fa29e0d5 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:02.134784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 935/1881] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:02.155382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd7126-c4fe-43d4-b6e7-9296ff6c3747 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:02.252945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd7126-c4fe-43d4-b6e7-9296ff6c3747 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:02.258385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:02.259414 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:02.259706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:02.260027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] PATH_INFO: `/v3/users/84f6af5b0b8b4ae9a730f27973d91b88` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:02.261254 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f6af5b0b8b4ae9a730f27973d91b88)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:02.263174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:02.314780 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f6af5b0b8b4ae9a730f27973d91b88'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:02.316553 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f6af5b0b8b4ae9a730f27973d91b88'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:02.326218 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f6af5b0b8b4ae9a730f27973d91b88'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:02.344790 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f6af5b0b8b4ae9a730f27973d91b88'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:02.345472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f6af5b0b8b4ae9a730f27973d91b88'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:02.363167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 947/1882] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:02 2022] DELETE /identity/v3/users/84f6af5b0b8b4ae9a730f27973d91b88 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:02.368431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0c290-24d8-4755-823a-86ae3839823f None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:02.368911 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0c290-24d8-4755-823a-86ae3839823f None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:02.368997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0c290-24d8-4755-823a-86ae3839823f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:02.387003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-2fc0c290-24d8-4755-823a-86ae3839823f None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:02.388399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-2fc0c290-24d8-4755-823a-86ae3839823f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:02.451502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 936/1883] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:02.462327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:02.463714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:02.523838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ade8d-0ad4-4695-a16b-4cfd684ffa2d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:02.527549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f56c2-2b8f-4f4c-873e-b7cc532bce12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:02.528237 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f56c2-2b8f-4f4c-873e-b7cc532bce12 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:02.528525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f56c2-2b8f-4f4c-873e-b7cc532bce12 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:02.528802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f56c2-2b8f-4f4c-873e-b7cc532bce12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:02.538218 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f56c2-2b8f-4f4c-873e-b7cc532bce12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:02.539743 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f56c2-2b8f-4f4c-873e-b7cc532bce12 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:02.549713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 948/1884] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:02.875977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0c290-24d8-4755-823a-86ae3839823f None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:02.877717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-2fc0c290-24d8-4755-823a-86ae3839823f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:02.898697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0c290-24d8-4755-823a-86ae3839823f None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:02.910038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:02.912704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:02.913255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:02.913792 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3 admin admin] PATH_INFO: `/v3/projects/792260608a9e4e05aafd375134e64c79` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:02.916328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=792260608a9e4e05aafd375134e64c79)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:02.918885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:02.991227 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '792260608a9e4e05aafd375134e64c79', 'request_id': 'req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:03.003573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '792260608a9e4e05aafd375134e64c79', 'request_id': 'req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:03.024268 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '792260608a9e4e05aafd375134e64c79', 'request_id': 'req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:03.045336 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3 admin admin] The token cache is being invalidate because project 792260608a9e4e05aafd375134e64c79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:03.047230 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:03.050289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6f723ec4-c9e1-428c-91d1-b863bd02f1e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:03.053926 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 937/1885] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:02 2022] DELETE /identity/v3/projects/792260608a9e4e05aafd375134e64c79 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:03.058776 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e7b97-0e42-4eac-a4df-6d6ed6b94858 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:03.059609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e7b97-0e42-4eac-a4df-6d6ed6b94858 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:03.060070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e7b97-0e42-4eac-a4df-6d6ed6b94858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:03.082348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c78e7b97-0e42-4eac-a4df-6d6ed6b94858 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:03.084325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c78e7b97-0e42-4eac-a4df-6d6ed6b94858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:03.126859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40d166-ebd4-4773-ab79-8488c27c3168 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:03.127256 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40d166-ebd4-4773-ab79-8488c27c3168 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:03.127683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40d166-ebd4-4773-ab79-8488c27c3168 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:03.130160 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 938/1886] 10.222.0.44 () {54 vars in 921 bytes} [Mon May 9 04:27:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:27:03.140049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520fa25-455a-4c74-ac38-51c249e9c9be None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:03.140342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520fa25-455a-4c74-ac38-51c249e9c9be None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:03.140620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520fa25-455a-4c74-ac38-51c249e9c9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:03.168804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-3520fa25-455a-4c74-ac38-51c249e9c9be None None] MFA Rules not processed for user `faf6107735424ff7901614953e7df403`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:03.170428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3520fa25-455a-4c74-ac38-51c249e9c9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:03.173821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 949/1887] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:03.184465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62a5d9-c7b2-4fc5-9bbc-00d4a1498af6 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:03.185620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62a5d9-c7b2-4fc5-9bbc-00d4a1498af6 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:03.185919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62a5d9-c7b2-4fc5-9bbc-00d4a1498af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:03.214931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-df62a5d9-c7b2-4fc5-9bbc-00d4a1498af6 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:03.217380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-df62a5d9-c7b2-4fc5-9bbc-00d4a1498af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:03.234039 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 950/1888] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:03.235479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62a5d9-c7b2-4fc5-9bbc-00d4a1498af6 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:03.250756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 939/1889] 10.222.0.44 () {58 vars in 1018 bytes} [Mon May 9 04:27:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:03.251856 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62a5d9-c7b2-4fc5-9bbc-00d4a1498af6 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:03.256697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18875638-0dbf-4021-ae3e-cd17c2e868f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:03.257417 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18875638-0dbf-4021-ae3e-cd17c2e868f1 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:03.257968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18875638-0dbf-4021-ae3e-cd17c2e868f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:03.258251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18875638-0dbf-4021-ae3e-cd17c2e868f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:03.258974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18875638-0dbf-4021-ae3e-cd17c2e868f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:03.259282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18875638-0dbf-4021-ae3e-cd17c2e868f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:03.260998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18875638-0dbf-4021-ae3e-cd17c2e868f1 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:03.263377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520fa25-455a-4c74-ac38-51c249e9c9be None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:03.264793 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3520fa25-455a-4c74-ac38-51c249e9c9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:03.274148 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 951/1890] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:27:03.278244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18875638-0dbf-4021-ae3e-cd17c2e868f1 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:03.279435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520fa25-455a-4c74-ac38-51c249e9c9be None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:03.285176 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141f0b3-41d7-4424-8c49-fd4d98dacc13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:03.286813 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141f0b3-41d7-4424-8c49-fd4d98dacc13 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:03.286983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141f0b3-41d7-4424-8c49-fd4d98dacc13 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:03.287338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141f0b3-41d7-4424-8c49-fd4d98dacc13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:03.289533 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18875638-0dbf-4021-ae3e-cd17c2e868f1 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:03.294210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5c979-39c2-48f8-817d-f145a05dfcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:03.294910 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5c979-39c2-48f8-817d-f145a05dfcd2 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:03.295704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5c979-39c2-48f8-817d-f145a05dfcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:03.295704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5c979-39c2-48f8-817d-f145a05dfcd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:03.296199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5c979-39c2-48f8-817d-f145a05dfcd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:03.298127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5c979-39c2-48f8-817d-f145a05dfcd2 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:03.299440 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141f0b3-41d7-4424-8c49-fd4d98dacc13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:03.302472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141f0b3-41d7-4424-8c49-fd4d98dacc13 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:03.314078 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 940/1891] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:27:03.333956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 952/1892] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:03 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:27:03.336442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141f0b3-41d7-4424-8c49-fd4d98dacc13 service nova] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:03.350290 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141f0b3-41d7-4424-8c49-fd4d98dacc13 service nova] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:03.413504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e349cb5-a718-4dca-8289-ceee4b0a119a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:03.414347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e349cb5-a718-4dca-8289-ceee4b0a119a admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:03.414724 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e349cb5-a718-4dca-8289-ceee4b0a119a admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:03.415097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e349cb5-a718-4dca-8289-ceee4b0a119a admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:03.415910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e349cb5-a718-4dca-8289-ceee4b0a119a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:03.417910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e349cb5-a718-4dca-8289-ceee4b0a119a admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:03.440017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 941/1893] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:03 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:03.442875 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5c979-39c2-48f8-817d-f145a05dfcd2 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:03.453082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5c979-39c2-48f8-817d-f145a05dfcd2 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:03.457672 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ab048-09bf-47df-a422-faac1cf8f273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:03.458911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ab048-09bf-47df-a422-faac1cf8f273 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:03.459420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ab048-09bf-47df-a422-faac1cf8f273 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:03.459789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ab048-09bf-47df-a422-faac1cf8f273 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:03.460866 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ab048-09bf-47df-a422-faac1cf8f273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:03.462541 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ab048-09bf-47df-a422-faac1cf8f273 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:03.472512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 953/1894] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:03.477199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e349cb5-a718-4dca-8289-ceee4b0a119a admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:03.486988 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e349cb5-a718-4dca-8289-ceee4b0a119a admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:03.492667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b8864-a1d6-40b3-a99b-67dc51dff60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:03.493528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b8864-a1d6-40b3-a99b-67dc51dff60f admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:03.493813 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b8864-a1d6-40b3-a99b-67dc51dff60f admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:03.494092 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b8864-a1d6-40b3-a99b-67dc51dff60f admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:03.494813 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b8864-a1d6-40b3-a99b-67dc51dff60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:03.496419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b8864-a1d6-40b3-a99b-67dc51dff60f admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:03.506946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 942/1895] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:03.511171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ab048-09bf-47df-a422-faac1cf8f273 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:03.521593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ab048-09bf-47df-a422-faac1cf8f273 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:03.526209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b06920-2734-4c09-b078-bbd5f646a74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:03.527079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b06920-2734-4c09-b078-bbd5f646a74c admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:03.527354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b06920-2734-4c09-b078-bbd5f646a74c admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:03.527619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b06920-2734-4c09-b078-bbd5f646a74c admin admin] PATH_INFO: `/v3/projects/d0e4adac41384d92b92e3011554cca8b/users/609e17667a2a4c519170cb96bd50a749/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:03.542612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b06920-2734-4c09-b078-bbd5f646a74c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0e4adac41384d92b92e3011554cca8b, user_id=609e17667a2a4c519170cb96bd50a749, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:03.545270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b06920-2734-4c09-b078-bbd5f646a74c admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:03.558479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 954/1896] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:03 2022] PUT /identity/v3/projects/d0e4adac41384d92b92e3011554cca8b/users/609e17667a2a4c519170cb96bd50a749/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:03.568680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d121957-fcdd-4d06-954d-f5d15d4a943b None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:03.568967 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d121957-fcdd-4d06-954d-f5d15d4a943b None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:03.569245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d121957-fcdd-4d06-954d-f5d15d4a943b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:03.595396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-3d121957-fcdd-4d06-954d-f5d15d4a943b None None] MFA Rules not processed for user `609e17667a2a4c519170cb96bd50a749`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:03.596649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3d121957-fcdd-4d06-954d-f5d15d4a943b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:03.658282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 943/1897] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:03.669781 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b06920-2734-4c09-b078-bbd5f646a74c admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:03.671441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-00b06920-2734-4c09-b078-bbd5f646a74c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:03.726119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b06920-2734-4c09-b078-bbd5f646a74c admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:03.729716 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33b766-5017-472f-98e4-859e1e018edd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:03.730361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33b766-5017-472f-98e4-859e1e018edd service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:03.730640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33b766-5017-472f-98e4-859e1e018edd service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:03.730913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33b766-5017-472f-98e4-859e1e018edd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:03.738912 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33b766-5017-472f-98e4-859e1e018edd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:03.741063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33b766-5017-472f-98e4-859e1e018edd service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:03.750624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 955/1898] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:06.162511 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d121957-fcdd-4d06-954d-f5d15d4a943b None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:06.164007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3d121957-fcdd-4d06-954d-f5d15d4a943b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:06.236024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d121957-fcdd-4d06-954d-f5d15d4a943b None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:06.241793 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:06.242856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:06.243188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:06.243419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] PATH_INFO: `/v3/users/968cae1bf670412e8cb22c5b2c0de987` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:06.244640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=968cae1bf670412e8cb22c5b2c0de987)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:06.246553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:06.314039 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '968cae1bf670412e8cb22c5b2c0de987'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:06.314789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '968cae1bf670412e8cb22c5b2c0de987'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:06.315449 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '968cae1bf670412e8cb22c5b2c0de987'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:06.327068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '968cae1bf670412e8cb22c5b2c0de987'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:06.337038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '968cae1bf670412e8cb22c5b2c0de987'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:06.356466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 944/1899] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:06 2022] DELETE /identity/v3/users/968cae1bf670412e8cb22c5b2c0de987 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:06.361575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e5c7b-f3eb-4f8b-bebe-f7e545fd092e None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:06.361575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e5c7b-f3eb-4f8b-bebe-f7e545fd092e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:06.361575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e5c7b-f3eb-4f8b-bebe-f7e545fd092e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:06.380757 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-533e5c7b-f3eb-4f8b-bebe-f7e545fd092e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:06.382338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-533e5c7b-f3eb-4f8b-bebe-f7e545fd092e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:06.467978 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 956/1900] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:06.479205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:06.480927 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:06.545241 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e57c26-1cc3-4849-958c-45734d15cbb1 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:06.549295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc3b17-792b-4a97-ae15-db520e96d39c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:06.549905 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc3b17-792b-4a97-ae15-db520e96d39c service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:06.550258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc3b17-792b-4a97-ae15-db520e96d39c service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:06.550534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc3b17-792b-4a97-ae15-db520e96d39c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:06.563133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc3b17-792b-4a97-ae15-db520e96d39c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:06.564743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc3b17-792b-4a97-ae15-db520e96d39c service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:06.576354 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 945/1901] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:06.940986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e5c7b-f3eb-4f8b-bebe-f7e545fd092e None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:06.943167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-533e5c7b-f3eb-4f8b-bebe-f7e545fd092e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:06.955789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e5c7b-f3eb-4f8b-bebe-f7e545fd092e None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:06.961136 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:06.962394 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:06.962782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:06.963190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] PATH_INFO: `/v3/projects/efd18d14203d49fd816186022e30e604` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:06.964669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=efd18d14203d49fd816186022e30e604)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:06.966691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:07.030870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efd18d14203d49fd816186022e30e604', 'request_id': 'req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.035973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efd18d14203d49fd816186022e30e604', 'request_id': 'req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.042806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efd18d14203d49fd816186022e30e604', 'request_id': 'req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.054260 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] The token cache is being invalidate because project efd18d14203d49fd816186022e30e604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:07.054704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.055212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.056833 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 957/1902] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:06 2022] DELETE /identity/v3/projects/efd18d14203d49fd816186022e30e604 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:07.060715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc3b17-792b-4a97-ae15-db520e96d39c service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:07.071109 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc3b17-792b-4a97-ae15-db520e96d39c service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:07.126882 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b4695-617b-4af3-b73d-72f2a422c322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:07.127753 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b4695-617b-4af3-b73d-72f2a422c322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:07.128077 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b4695-617b-4af3-b73d-72f2a422c322 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:07.128350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b4695-617b-4af3-b73d-72f2a422c322 admin admin] PATH_INFO: `/v3/users/251cc65a865046c8a0770d7169f98f5c` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:07.129506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b4695-617b-4af3-b73d-72f2a422c322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=251cc65a865046c8a0770d7169f98f5c)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:07.131986 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b4695-617b-4af3-b73d-72f2a422c322 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:07.174050 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6f5b4695-617b-4af3-b73d-72f2a422c322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '251cc65a865046c8a0770d7169f98f5c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.174697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6f5b4695-617b-4af3-b73d-72f2a422c322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '251cc65a865046c8a0770d7169f98f5c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.175326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6f5b4695-617b-4af3-b73d-72f2a422c322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '251cc65a865046c8a0770d7169f98f5c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.182800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6f5b4695-617b-4af3-b73d-72f2a422c322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '251cc65a865046c8a0770d7169f98f5c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.189608 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6f5b4695-617b-4af3-b73d-72f2a422c322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '251cc65a865046c8a0770d7169f98f5c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.206513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 946/1903] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:07 2022] DELETE /identity/v3/users/251cc65a865046c8a0770d7169f98f5c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:27:07.477426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:07.479765 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:07.497221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39edd9-dce0-4b01-a7fb-9f316ad7da7c admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:07.571854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2924b8-e78a-41a7-b995-6c088a152932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:07.572940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2924b8-e78a-41a7-b995-6c088a152932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:07.573417 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2924b8-e78a-41a7-b995-6c088a152932 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:07.573762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2924b8-e78a-41a7-b995-6c088a152932 admin admin] PATH_INFO: `/v3/projects/bdf81c906ff94ef08d709ec14305418f` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:07.575724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2924b8-e78a-41a7-b995-6c088a152932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdf81c906ff94ef08d709ec14305418f)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:07.579141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2924b8-e78a-41a7-b995-6c088a152932 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:07.631986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2b2924b8-e78a-41a7-b995-6c088a152932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdf81c906ff94ef08d709ec14305418f', 'request_id': 'req-2b2924b8-e78a-41a7-b995-6c088a152932'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.637676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2b2924b8-e78a-41a7-b995-6c088a152932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdf81c906ff94ef08d709ec14305418f', 'request_id': 'req-2b2924b8-e78a-41a7-b995-6c088a152932'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.645877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2b2924b8-e78a-41a7-b995-6c088a152932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdf81c906ff94ef08d709ec14305418f', 'request_id': 'req-2b2924b8-e78a-41a7-b995-6c088a152932'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.662630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2b2924b8-e78a-41a7-b995-6c088a152932 admin admin] The token cache is being invalidate because project bdf81c906ff94ef08d709ec14305418f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:07.663139 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2b2924b8-e78a-41a7-b995-6c088a152932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.663871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2b2924b8-e78a-41a7-b995-6c088a152932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:07.665842 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 958/1904] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:07 2022] DELETE /identity/v3/projects/bdf81c906ff94ef08d709ec14305418f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:27:07.670615 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518f206-45ca-40be-9057-d960dffac8df None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:07.670909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518f206-45ca-40be-9057-d960dffac8df None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:07.671357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518f206-45ca-40be-9057-d960dffac8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:07.696679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-6518f206-45ca-40be-9057-d960dffac8df None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:07.698191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-6518f206-45ca-40be-9057-d960dffac8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:07.787096 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 947/1905] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:07.794951 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b262fc5-1eb5-435d-8be3-cd01285a8bca None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:07.795249 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b262fc5-1eb5-435d-8be3-cd01285a8bca None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:07.795922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b262fc5-1eb5-435d-8be3-cd01285a8bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:07.818071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-7b262fc5-1eb5-435d-8be3-cd01285a8bca None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:07.819774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-7b262fc5-1eb5-435d-8be3-cd01285a8bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:07.831817 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 959/1906] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:07.836535 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518f206-45ca-40be-9057-d960dffac8df None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:07.846282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518f206-45ca-40be-9057-d960dffac8df None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:07.851677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab9d33-3c68-4f6b-98ce-91ab5dab1c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:07.852396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab9d33-3c68-4f6b-98ce-91ab5dab1c0e admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:07.852682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab9d33-3c68-4f6b-98ce-91ab5dab1c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:07.853089 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab9d33-3c68-4f6b-98ce-91ab5dab1c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:07.854072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab9d33-3c68-4f6b-98ce-91ab5dab1c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:07.854607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab9d33-3c68-4f6b-98ce-91ab5dab1c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:07.857359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab9d33-3c68-4f6b-98ce-91ab5dab1c0e admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:07.870935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 948/1907] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:27:07.875309 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b262fc5-1eb5-435d-8be3-cd01285a8bca None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:07.885997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b262fc5-1eb5-435d-8be3-cd01285a8bca None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:07.891572 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902d83f-e176-40ca-b2e5-9179d255c03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:07.892575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902d83f-e176-40ca-b2e5-9179d255c03c admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:07.892982 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902d83f-e176-40ca-b2e5-9179d255c03c admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:07.893251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902d83f-e176-40ca-b2e5-9179d255c03c admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:07.894050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a902d83f-e176-40ca-b2e5-9179d255c03c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:07.896007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a902d83f-e176-40ca-b2e5-9179d255c03c admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:07.930928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 960/1908] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:07 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:07.932130 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab9d33-3c68-4f6b-98ce-91ab5dab1c0e admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:07.941955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab9d33-3c68-4f6b-98ce-91ab5dab1c0e admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:07.999918 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fd765-b74c-4c81-8de8-931e1278a6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:08.000592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fd765-b74c-4c81-8de8-931e1278a6d6 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:08.000875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fd765-b74c-4c81-8de8-931e1278a6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:08.001149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fd765-b74c-4c81-8de8-931e1278a6d6 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:08.001842 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fd765-b74c-4c81-8de8-931e1278a6d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:08.003715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fd765-b74c-4c81-8de8-931e1278a6d6 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:08.024028 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 949/1909] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:07 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:08.028027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902d83f-e176-40ca-b2e5-9179d255c03c admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:08.040347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902d83f-e176-40ca-b2e5-9179d255c03c admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:08.045068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543eef0-3f54-4c4e-8182-4a160d2932f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:08.045735 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543eef0-3f54-4c4e-8182-4a160d2932f7 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:08.046021 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543eef0-3f54-4c4e-8182-4a160d2932f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:08.046296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543eef0-3f54-4c4e-8182-4a160d2932f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:08.047081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543eef0-3f54-4c4e-8182-4a160d2932f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:08.048712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543eef0-3f54-4c4e-8182-4a160d2932f7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:08.059465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 961/1910] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:08.063368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fd765-b74c-4c81-8de8-931e1278a6d6 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:08.074034 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fd765-b74c-4c81-8de8-931e1278a6d6 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:08.078941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34231628-670a-4200-ad2a-5b26cbf77c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:08.079442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34231628-670a-4200-ad2a-5b26cbf77c29 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:08.079723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34231628-670a-4200-ad2a-5b26cbf77c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:08.080044 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34231628-670a-4200-ad2a-5b26cbf77c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:08.080771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34231628-670a-4200-ad2a-5b26cbf77c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:08.082347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34231628-670a-4200-ad2a-5b26cbf77c29 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:08.094543 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 950/1911] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:08.099017 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543eef0-3f54-4c4e-8182-4a160d2932f7 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:08.109561 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543eef0-3f54-4c4e-8182-4a160d2932f7 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:08.116133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed7f59-9e80-4e95-bedf-e60e076a2e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:08.117038 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed7f59-9e80-4e95-bedf-e60e076a2e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:08.117624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed7f59-9e80-4e95-bedf-e60e076a2e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:08.117906 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed7f59-9e80-4e95-bedf-e60e076a2e2d admin admin] PATH_INFO: `/v3/projects/d209ebca9c7a452bab3305415aec7116/users/2346ed4941fa45a3bb0578bde8581f27/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:08.136536 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed7f59-9e80-4e95-bedf-e60e076a2e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d209ebca9c7a452bab3305415aec7116, user_id=2346ed4941fa45a3bb0578bde8581f27, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:08.139196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed7f59-9e80-4e95-bedf-e60e076a2e2d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:08.162174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 962/1912] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:08 2022] PUT /identity/v3/projects/d209ebca9c7a452bab3305415aec7116/users/2346ed4941fa45a3bb0578bde8581f27/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:08.175099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671996d4-cbff-4b76-b38b-8a980e0b8eb6 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:08.175585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671996d4-cbff-4b76-b38b-8a980e0b8eb6 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:08.175674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671996d4-cbff-4b76-b38b-8a980e0b8eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:08.202387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-671996d4-cbff-4b76-b38b-8a980e0b8eb6 None None] MFA Rules not processed for user `2346ed4941fa45a3bb0578bde8581f27`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:08.203604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-671996d4-cbff-4b76-b38b-8a980e0b8eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:08.267329 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 951/1913] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:08.272055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed7f59-9e80-4e95-bedf-e60e076a2e2d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:08.330042 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed7f59-9e80-4e95-bedf-e60e076a2e2d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:08.334511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa303167-c1b9-4f37-99dd-52fd4bf95faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:08.335158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa303167-c1b9-4f37-99dd-52fd4bf95faa admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:08.335436 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa303167-c1b9-4f37-99dd-52fd4bf95faa admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:08.335993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa303167-c1b9-4f37-99dd-52fd4bf95faa admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:08.336680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa303167-c1b9-4f37-99dd-52fd4bf95faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:08.338553 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa303167-c1b9-4f37-99dd-52fd4bf95faa admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:08.362831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 963/1914] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:08 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:08.367607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671996d4-cbff-4b76-b38b-8a980e0b8eb6 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:08.378130 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671996d4-cbff-4b76-b38b-8a980e0b8eb6 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:08.433972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa6e38-f394-4715-a29b-d65637415a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:08.434642 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa6e38-f394-4715-a29b-d65637415a3d admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:08.434924 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa6e38-f394-4715-a29b-d65637415a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:08.435198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa6e38-f394-4715-a29b-d65637415a3d admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:08.435913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa6e38-f394-4715-a29b-d65637415a3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:08.437796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa6e38-f394-4715-a29b-d65637415a3d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:08.457220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 952/1915] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:08 2022] POST /identity/v3/users => generated 298 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:08.462315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa303167-c1b9-4f37-99dd-52fd4bf95faa admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:08.472476 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa303167-c1b9-4f37-99dd-52fd4bf95faa admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:08.477625 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9916ea-6418-438c-a0e0-94677971b97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:08.478401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9916ea-6418-438c-a0e0-94677971b97a admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:08.478690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9916ea-6418-438c-a0e0-94677971b97a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:08.478966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9916ea-6418-438c-a0e0-94677971b97a admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:08.479694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9916ea-6418-438c-a0e0-94677971b97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:08.481317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9916ea-6418-438c-a0e0-94677971b97a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:08.492628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 964/1916] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:08.496866 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa6e38-f394-4715-a29b-d65637415a3d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:08.507242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa6e38-f394-4715-a29b-d65637415a3d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:08.511725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaeb915-3ab7-4b34-8f40-f79060e9b241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:08.512633 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaeb915-3ab7-4b34-8f40-f79060e9b241 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:08.512912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaeb915-3ab7-4b34-8f40-f79060e9b241 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:08.513186 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaeb915-3ab7-4b34-8f40-f79060e9b241 admin admin] PATH_INFO: `/v3/projects/84c3842cf96d4fc9925533012385c381/users/d1d7996cd04d442395e910e81ca71993/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:08.527143 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaeb915-3ab7-4b34-8f40-f79060e9b241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c3842cf96d4fc9925533012385c381, user_id=d1d7996cd04d442395e910e81ca71993, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:08.529863 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaeb915-3ab7-4b34-8f40-f79060e9b241 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:08.541832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 953/1917] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:08 2022] PUT /identity/v3/projects/84c3842cf96d4fc9925533012385c381/users/d1d7996cd04d442395e910e81ca71993/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:08.545837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9916ea-6418-438c-a0e0-94677971b97a admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:08.556462 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9916ea-6418-438c-a0e0-94677971b97a admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:08.612272 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157e806-245a-4ccf-befd-452d91503c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:08.612976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157e806-245a-4ccf-befd-452d91503c48 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:08.613259 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157e806-245a-4ccf-befd-452d91503c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:08.613535 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157e806-245a-4ccf-befd-452d91503c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:08.614798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157e806-245a-4ccf-befd-452d91503c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:08.616417 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157e806-245a-4ccf-befd-452d91503c48 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:08.625908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 965/1918] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:27:08.630713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaeb915-3ab7-4b34-8f40-f79060e9b241 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:08.640505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaeb915-3ab7-4b34-8f40-f79060e9b241 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:08.644958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32aff8d-f766-43ed-849a-0e15a962b739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:08.645876 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aff8d-f766-43ed-849a-0e15a962b739 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:08.646168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aff8d-f766-43ed-849a-0e15a962b739 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:08.646439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aff8d-f766-43ed-849a-0e15a962b739 admin admin] PATH_INFO: `/v3/projects/84c3842cf96d4fc9925533012385c381/users/d1d7996cd04d442395e910e81ca71993/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:08.648380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32aff8d-f766-43ed-849a-0e15a962b739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c3842cf96d4fc9925533012385c381, user_id=d1d7996cd04d442395e910e81ca71993, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:08.650958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32aff8d-f766-43ed-849a-0e15a962b739 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:08.660847 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 954/1919] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:08 2022] PUT /identity/v3/projects/84c3842cf96d4fc9925533012385c381/users/d1d7996cd04d442395e910e81ca71993/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:08.670971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944a81e-f6e2-408f-8bde-121322347ee9 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:08.671263 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944a81e-f6e2-408f-8bde-121322347ee9 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:08.671543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944a81e-f6e2-408f-8bde-121322347ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:08.697813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2944a81e-f6e2-408f-8bde-121322347ee9 None None] MFA Rules not processed for user `d1d7996cd04d442395e910e81ca71993`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:08.699014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2944a81e-f6e2-408f-8bde-121322347ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:08.766737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 966/1920] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:08.777388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32aff8d-f766-43ed-849a-0e15a962b739 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:08.779123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e32aff8d-f766-43ed-849a-0e15a962b739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:08.837029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32aff8d-f766-43ed-849a-0e15a962b739 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:08.840489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7be8c8-71c9-414f-8965-27d4ef467342 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:08.841135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7be8c8-71c9-414f-8965-27d4ef467342 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:08.841420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7be8c8-71c9-414f-8965-27d4ef467342 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:08.841689 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7be8c8-71c9-414f-8965-27d4ef467342 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:08.899275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7be8c8-71c9-414f-8965-27d4ef467342 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:08.900769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7be8c8-71c9-414f-8965-27d4ef467342 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:08.909764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 955/1921] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:16.804687 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944a81e-f6e2-408f-8bde-121322347ee9 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:16.806119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2944a81e-f6e2-408f-8bde-121322347ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:16.870063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944a81e-f6e2-408f-8bde-121322347ee9 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:16.874959 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:16.876025 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:16.876368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:16.876642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] PATH_INFO: `/v3/users/c28066d9a3ab4d58aa9e950abb9ec2c9` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:16.877847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c28066d9a3ab4d58aa9e950abb9ec2c9)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:16.880844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:16.937983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c28066d9a3ab4d58aa9e950abb9ec2c9'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:16.938651 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c28066d9a3ab4d58aa9e950abb9ec2c9'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:16.948258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c28066d9a3ab4d58aa9e950abb9ec2c9'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:16.955506 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c28066d9a3ab4d58aa9e950abb9ec2c9'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:16.959640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c28066d9a3ab4d58aa9e950abb9ec2c9'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:16.977506 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 967/1922] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:16 2022] DELETE /identity/v3/users/c28066d9a3ab4d58aa9e950abb9ec2c9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:16.981636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d12fc-8abf-4607-9401-238b62b8bcbc None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:16.982117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d12fc-8abf-4607-9401-238b62b8bcbc None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:16.982275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d12fc-8abf-4607-9401-238b62b8bcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:17.009451 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-5b8d12fc-8abf-4607-9401-238b62b8bcbc None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:17.010662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5b8d12fc-8abf-4607-9401-238b62b8bcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:17.079138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 956/1923] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:17.093936 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:17.095429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:17.162234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8011c-e251-4e8d-95d8-498d988ef8d0 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:17.166781 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbd9c3-fc50-4ba8-a28f-e9acd6396759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:17.167440 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbd9c3-fc50-4ba8-a28f-e9acd6396759 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:17.167720 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbd9c3-fc50-4ba8-a28f-e9acd6396759 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:17.168021 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbd9c3-fc50-4ba8-a28f-e9acd6396759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:17.183314 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbd9c3-fc50-4ba8-a28f-e9acd6396759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:17.184956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbd9c3-fc50-4ba8-a28f-e9acd6396759 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:17.195703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 968/1924] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:27:17.533592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d12fc-8abf-4607-9401-238b62b8bcbc None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:17.534987 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5b8d12fc-8abf-4607-9401-238b62b8bcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:17.546918 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d12fc-8abf-4607-9401-238b62b8bcbc None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:17.552649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:17.553524 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:17.553820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:17.554073 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] PATH_INFO: `/v3/projects/aee7fe01f201408e8949d3405fe44586` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:17.555259 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aee7fe01f201408e8949d3405fe44586)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:17.557283 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:17.602845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aee7fe01f201408e8949d3405fe44586', 'request_id': 'req-930eedd5-9b65-4904-b8a0-f5e923fb1695'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:17.608786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aee7fe01f201408e8949d3405fe44586', 'request_id': 'req-930eedd5-9b65-4904-b8a0-f5e923fb1695'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:17.614539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aee7fe01f201408e8949d3405fe44586', 'request_id': 'req-930eedd5-9b65-4904-b8a0-f5e923fb1695'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:17.627535 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] The token cache is being invalidate because project aee7fe01f201408e8949d3405fe44586 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:17.627833 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:17.628762 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:17.631020 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 957/1925] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:17 2022] DELETE /identity/v3/projects/aee7fe01f201408e8949d3405fe44586 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:17.635251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbd9c3-fc50-4ba8-a28f-e9acd6396759 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:17.647844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbd9c3-fc50-4ba8-a28f-e9acd6396759 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:17.707718 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033748f-5f60-4786-81cd-ece94c688f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:17.708826 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033748f-5f60-4786-81cd-ece94c688f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:17.709111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033748f-5f60-4786-81cd-ece94c688f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:17.709387 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033748f-5f60-4786-81cd-ece94c688f5f admin admin] PATH_INFO: `/v3/users/597169257ff94108832959dc37f27164` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:17.710670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033748f-5f60-4786-81cd-ece94c688f5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=597169257ff94108832959dc37f27164)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:17.712941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033748f-5f60-4786-81cd-ece94c688f5f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:17.769119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6033748f-5f60-4786-81cd-ece94c688f5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '597169257ff94108832959dc37f27164'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:17.770659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6033748f-5f60-4786-81cd-ece94c688f5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '597169257ff94108832959dc37f27164'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:17.780974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6033748f-5f60-4786-81cd-ece94c688f5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '597169257ff94108832959dc37f27164'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:17.790161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6033748f-5f60-4786-81cd-ece94c688f5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '597169257ff94108832959dc37f27164'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:17.790831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6033748f-5f60-4786-81cd-ece94c688f5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '597169257ff94108832959dc37f27164'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:17.812113 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 969/1926] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:17 2022] DELETE /identity/v3/users/597169257ff94108832959dc37f27164 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:18.081374 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:18.083551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:18.097725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930eedd5-9b65-4904-b8a0-f5e923fb1695 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:18.153355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c888a29-c7e0-498e-879b-1653d6424518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:18.154216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c888a29-c7e0-498e-879b-1653d6424518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:18.154492 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c888a29-c7e0-498e-879b-1653d6424518 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:18.154764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c888a29-c7e0-498e-879b-1653d6424518 admin admin] PATH_INFO: `/v3/projects/8c10981ed3e443fd99a7241783c95882` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:18.155950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c888a29-c7e0-498e-879b-1653d6424518 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c10981ed3e443fd99a7241783c95882)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:18.157912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c888a29-c7e0-498e-879b-1653d6424518 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:18.201444 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2c888a29-c7e0-498e-879b-1653d6424518 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c10981ed3e443fd99a7241783c95882', 'request_id': 'req-2c888a29-c7e0-498e-879b-1653d6424518'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:18.207475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2c888a29-c7e0-498e-879b-1653d6424518 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c10981ed3e443fd99a7241783c95882', 'request_id': 'req-2c888a29-c7e0-498e-879b-1653d6424518'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:18.213488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2c888a29-c7e0-498e-879b-1653d6424518 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c10981ed3e443fd99a7241783c95882', 'request_id': 'req-2c888a29-c7e0-498e-879b-1653d6424518'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:18.225974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2c888a29-c7e0-498e-879b-1653d6424518 admin admin] The token cache is being invalidate because project 8c10981ed3e443fd99a7241783c95882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:18.226265 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2c888a29-c7e0-498e-879b-1653d6424518 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:18.226877 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2c888a29-c7e0-498e-879b-1653d6424518 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:18.228594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 958/1927] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:18 2022] DELETE /identity/v3/projects/8c10981ed3e443fd99a7241783c95882 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:18.233571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834dfd5-20cb-482b-a532-32566bde4d54 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:18.233882 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834dfd5-20cb-482b-a532-32566bde4d54 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:18.234158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834dfd5-20cb-482b-a532-32566bde4d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:18.252225 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-5834dfd5-20cb-482b-a532-32566bde4d54 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:18.253448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5834dfd5-20cb-482b-a532-32566bde4d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:18.315107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 970/1928] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:18.323465 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c22522-cd84-49ac-b6c0-dab258c342e2 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:18.323820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c22522-cd84-49ac-b6c0-dab258c342e2 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:18.324209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c22522-cd84-49ac-b6c0-dab258c342e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:18.342120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-70c22522-cd84-49ac-b6c0-dab258c342e2 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:18.343330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-70c22522-cd84-49ac-b6c0-dab258c342e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:18.352406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 959/1929] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:18.356584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834dfd5-20cb-482b-a532-32566bde4d54 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:18.367835 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834dfd5-20cb-482b-a532-32566bde4d54 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:18.374385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b464d-8585-48a0-9db2-72816faa102c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:18.375059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b464d-8585-48a0-9db2-72816faa102c admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:18.375347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b464d-8585-48a0-9db2-72816faa102c admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:18.375629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b464d-8585-48a0-9db2-72816faa102c admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:18.376394 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b464d-8585-48a0-9db2-72816faa102c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:18.376706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b464d-8585-48a0-9db2-72816faa102c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:18.378310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b464d-8585-48a0-9db2-72816faa102c admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:18.391332 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 971/1930] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:27:18.396287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c22522-cd84-49ac-b6c0-dab258c342e2 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:18.405953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c22522-cd84-49ac-b6c0-dab258c342e2 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:18.415217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee386e0-4b66-4387-a870-25ced57da98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:18.416302 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee386e0-4b66-4387-a870-25ced57da98a admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:18.416584 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee386e0-4b66-4387-a870-25ced57da98a admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:18.416858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee386e0-4b66-4387-a870-25ced57da98a admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:18.417723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee386e0-4b66-4387-a870-25ced57da98a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:18.419626 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee386e0-4b66-4387-a870-25ced57da98a admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:18.450032 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 960/1931] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:18 2022] POST /identity/v3/projects => generated 357 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:18.458617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b464d-8585-48a0-9db2-72816faa102c admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:18.470246 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b464d-8585-48a0-9db2-72816faa102c admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:18.526791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8974f-45db-4af7-8303-9feba69ba918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:18.527471 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8974f-45db-4af7-8303-9feba69ba918 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:18.527752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8974f-45db-4af7-8303-9feba69ba918 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:18.528070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8974f-45db-4af7-8303-9feba69ba918 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:18.528816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8974f-45db-4af7-8303-9feba69ba918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:18.530714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8974f-45db-4af7-8303-9feba69ba918 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:18.552581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 972/1932] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:18 2022] POST /identity/v3/users => generated 279 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:18.556592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee386e0-4b66-4387-a870-25ced57da98a admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:18.566770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee386e0-4b66-4387-a870-25ced57da98a admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:18.572308 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf6e0b-e3e7-477f-8d93-22193f4e8b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:18.572971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf6e0b-e3e7-477f-8d93-22193f4e8b5e admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:18.573251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf6e0b-e3e7-477f-8d93-22193f4e8b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:18.573523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf6e0b-e3e7-477f-8d93-22193f4e8b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:18.574297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf6e0b-e3e7-477f-8d93-22193f4e8b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:18.575814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf6e0b-e3e7-477f-8d93-22193f4e8b5e admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:18.587089 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 961/1933] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:18.591134 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8974f-45db-4af7-8303-9feba69ba918 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:18.601489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8974f-45db-4af7-8303-9feba69ba918 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:18.606372 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dedd1ec-834b-4058-a1ce-95daa83c985d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:18.607090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedd1ec-834b-4058-a1ce-95daa83c985d admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:18.607374 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedd1ec-834b-4058-a1ce-95daa83c985d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:18.607652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedd1ec-834b-4058-a1ce-95daa83c985d admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:18.608421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dedd1ec-834b-4058-a1ce-95daa83c985d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:18.610021 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dedd1ec-834b-4058-a1ce-95daa83c985d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:18.623420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 973/1934] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:27:18.624620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf6e0b-e3e7-477f-8d93-22193f4e8b5e admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:18.635616 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf6e0b-e3e7-477f-8d93-22193f4e8b5e admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:18.640236 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee871432-34d1-49a7-b3e7-d77b96051f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:18.641155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee871432-34d1-49a7-b3e7-d77b96051f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:18.641435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee871432-34d1-49a7-b3e7-d77b96051f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:18.641703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee871432-34d1-49a7-b3e7-d77b96051f68 admin admin] PATH_INFO: `/v3/projects/4d5d3a868f1044248d30b5f8604f631b/users/7cf9bb405a0e4dd597066240310990f3/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:18.656815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee871432-34d1-49a7-b3e7-d77b96051f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5d3a868f1044248d30b5f8604f631b, user_id=7cf9bb405a0e4dd597066240310990f3, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:18.659428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee871432-34d1-49a7-b3e7-d77b96051f68 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:18.671688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 962/1935] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:18 2022] PUT /identity/v3/projects/4d5d3a868f1044248d30b5f8604f631b/users/7cf9bb405a0e4dd597066240310990f3/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:27:18.682408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d4fcb-22a2-4d6a-a400-5e6d75f2d2fb None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:18.682701 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d4fcb-22a2-4d6a-a400-5e6d75f2d2fb None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:18.682979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d4fcb-22a2-4d6a-a400-5e6d75f2d2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:18.709642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-a42d4fcb-22a2-4d6a-a400-5e6d75f2d2fb None None] MFA Rules not processed for user `7cf9bb405a0e4dd597066240310990f3`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:18.711087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a42d4fcb-22a2-4d6a-a400-5e6d75f2d2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:18.773216 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 974/1936] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:18 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:18.777706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee871432-34d1-49a7-b3e7-d77b96051f68 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:18.843289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee871432-34d1-49a7-b3e7-d77b96051f68 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:18.847920 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f19ea-f9d8-4495-86ab-9badfc73489a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:18.849271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f19ea-f9d8-4495-86ab-9badfc73489a admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:18.849551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f19ea-f9d8-4495-86ab-9badfc73489a admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:18.849830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f19ea-f9d8-4495-86ab-9badfc73489a admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:18.850579 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f19ea-f9d8-4495-86ab-9badfc73489a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:18.852477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f19ea-f9d8-4495-86ab-9badfc73489a admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:18.878464 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 963/1937] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:18 2022] POST /identity/v3/projects => generated 355 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:18.882398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d4fcb-22a2-4d6a-a400-5e6d75f2d2fb None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:18.892751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d4fcb-22a2-4d6a-a400-5e6d75f2d2fb None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:18.947475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786db2b3-eea2-49f6-9c89-f7ecc7944cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:18.948194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786db2b3-eea2-49f6-9c89-f7ecc7944cbc admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:18.948479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786db2b3-eea2-49f6-9c89-f7ecc7944cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:18.948755 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786db2b3-eea2-49f6-9c89-f7ecc7944cbc admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:18.949465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786db2b3-eea2-49f6-9c89-f7ecc7944cbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:18.951496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786db2b3-eea2-49f6-9c89-f7ecc7944cbc admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:18.974509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 975/1938] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:18 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:18.978643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f19ea-f9d8-4495-86ab-9badfc73489a admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:18.989000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f19ea-f9d8-4495-86ab-9badfc73489a admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:18.993654 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3bf60-b5eb-4d10-a6f1-78a388647447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:18.994312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3bf60-b5eb-4d10-a6f1-78a388647447 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:18.994594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3bf60-b5eb-4d10-a6f1-78a388647447 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:18.994868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3bf60-b5eb-4d10-a6f1-78a388647447 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:18.995704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3bf60-b5eb-4d10-a6f1-78a388647447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:18.997308 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3bf60-b5eb-4d10-a6f1-78a388647447 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:19.008643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 964/1939] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:19.012849 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786db2b3-eea2-49f6-9c89-f7ecc7944cbc admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:19.023490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786db2b3-eea2-49f6-9c89-f7ecc7944cbc admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:19.028067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e2c02-eca4-4325-9c21-28014ce5a7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:19.028944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e2c02-eca4-4325-9c21-28014ce5a7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:19.029223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e2c02-eca4-4325-9c21-28014ce5a7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:19.029491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e2c02-eca4-4325-9c21-28014ce5a7f7 admin admin] PATH_INFO: `/v3/projects/dd49b016a4464610b94e18580f48e004/users/d6ec1c4f5f1f4e598908b22381743beb/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:19.045006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e2c02-eca4-4325-9c21-28014ce5a7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd49b016a4464610b94e18580f48e004, user_id=d6ec1c4f5f1f4e598908b22381743beb, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:19.047777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e2c02-eca4-4325-9c21-28014ce5a7f7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:19.061951 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 976/1940] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:19 2022] PUT /identity/v3/projects/dd49b016a4464610b94e18580f48e004/users/d6ec1c4f5f1f4e598908b22381743beb/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:27:19.065483 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3bf60-b5eb-4d10-a6f1-78a388647447 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:19.076345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3bf60-b5eb-4d10-a6f1-78a388647447 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:19.134964 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd228f1-3a9a-4511-9aeb-29eba380dd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:19.135639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd228f1-3a9a-4511-9aeb-29eba380dd0d admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:19.135953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd228f1-3a9a-4511-9aeb-29eba380dd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:19.136231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd228f1-3a9a-4511-9aeb-29eba380dd0d admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:19.136949 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd228f1-3a9a-4511-9aeb-29eba380dd0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:19.138527 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd228f1-3a9a-4511-9aeb-29eba380dd0d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:19.148518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 965/1941] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:19.152548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e2c02-eca4-4325-9c21-28014ce5a7f7 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:19.163069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e2c02-eca4-4325-9c21-28014ce5a7f7 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:19.167605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fe57e-58d8-4640-b804-6146853885d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:19.168525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fe57e-58d8-4640-b804-6146853885d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:19.168806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fe57e-58d8-4640-b804-6146853885d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:19.169076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fe57e-58d8-4640-b804-6146853885d8 admin admin] PATH_INFO: `/v3/projects/dd49b016a4464610b94e18580f48e004/users/d6ec1c4f5f1f4e598908b22381743beb/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:19.171009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fe57e-58d8-4640-b804-6146853885d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd49b016a4464610b94e18580f48e004, user_id=d6ec1c4f5f1f4e598908b22381743beb, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:19.173645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fe57e-58d8-4640-b804-6146853885d8 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:19.183581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 977/1942] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:19 2022] PUT /identity/v3/projects/dd49b016a4464610b94e18580f48e004/users/d6ec1c4f5f1f4e598908b22381743beb/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:19.194025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1e9bf-c19c-456e-bd4b-f4d165bd89e0 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:19.194321 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1e9bf-c19c-456e-bd4b-f4d165bd89e0 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:19.194604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1e9bf-c19c-456e-bd4b-f4d165bd89e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:19.221252 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-16e1e9bf-c19c-456e-bd4b-f4d165bd89e0 None None] MFA Rules not processed for user `d6ec1c4f5f1f4e598908b22381743beb`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:19.222453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-16e1e9bf-c19c-456e-bd4b-f4d165bd89e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:19.290151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 966/1943] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:19 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:19.302355 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fe57e-58d8-4640-b804-6146853885d8 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:19.304074 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c04fe57e-58d8-4640-b804-6146853885d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:19.358880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fe57e-58d8-4640-b804-6146853885d8 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:19.362631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebac3c1-b5f0-4e6a-8ccc-7b8ff2850a55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:19.363280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebac3c1-b5f0-4e6a-8ccc-7b8ff2850a55 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:19.363557 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebac3c1-b5f0-4e6a-8ccc-7b8ff2850a55 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:19.363880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebac3c1-b5f0-4e6a-8ccc-7b8ff2850a55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:19.374118 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebac3c1-b5f0-4e6a-8ccc-7b8ff2850a55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:19.375534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebac3c1-b5f0-4e6a-8ccc-7b8ff2850a55 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:19.385914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 978/1944] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:19 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:19.458898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1e9bf-c19c-456e-bd4b-f4d165bd89e0 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:19.469232 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1e9bf-c19c-456e-bd4b-f4d165bd89e0 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:19.473696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868a74a-263b-4eef-a624-dfcc92f420ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:19.474488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868a74a-263b-4eef-a624-dfcc92f420ec service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:19.474778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868a74a-263b-4eef-a624-dfcc92f420ec service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:19.475069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868a74a-263b-4eef-a624-dfcc92f420ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:19.535983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868a74a-263b-4eef-a624-dfcc92f420ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:19.537428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868a74a-263b-4eef-a624-dfcc92f420ec service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:19.546523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 967/1945] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:19 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:21.550259 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebac3c1-b5f0-4e6a-8ccc-7b8ff2850a55 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:21.559043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebac3c1-b5f0-4e6a-8ccc-7b8ff2850a55 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:21.562624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bed452-4935-4f75-9660-7e1f691d90f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:21.563255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bed452-4935-4f75-9660-7e1f691d90f1 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:21.563533 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bed452-4935-4f75-9660-7e1f691d90f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:21.563803 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bed452-4935-4f75-9660-7e1f691d90f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:21.565543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a6bed452-4935-4f75-9660-7e1f691d90f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:21.629081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bed452-4935-4f75-9660-7e1f691d90f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:21.630507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bed452-4935-4f75-9660-7e1f691d90f1 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:21.640611 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 979/1946] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:25.518250 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868a74a-263b-4eef-a624-dfcc92f420ec service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:25.519682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a868a74a-263b-4eef-a624-dfcc92f420ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:25.607491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868a74a-263b-4eef-a624-dfcc92f420ec service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:25.611930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:25.613017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:25.613298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:25.613565 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] PATH_INFO: `/v3/users/5c29159f83644a51a86604fa1df2950a` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:25.614733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c29159f83644a51a86604fa1df2950a)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:25.617332 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:25.664569 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c29159f83644a51a86604fa1df2950a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:25.665301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c29159f83644a51a86604fa1df2950a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:25.665969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c29159f83644a51a86604fa1df2950a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:25.673585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c29159f83644a51a86604fa1df2950a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:25.681316 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c29159f83644a51a86604fa1df2950a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:25.699440 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 968/1947] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:25 2022] DELETE /identity/v3/users/5c29159f83644a51a86604fa1df2950a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:25.703934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462afcbb-b719-4433-a8db-3e19658ad358 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:25.704421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462afcbb-b719-4433-a8db-3e19658ad358 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:25.704618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462afcbb-b719-4433-a8db-3e19658ad358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:25.722296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-462afcbb-b719-4433-a8db-3e19658ad358 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:25.723603 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-462afcbb-b719-4433-a8db-3e19658ad358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:25.790483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 980/1948] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:25.801547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:25.803024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:25.861044 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092d3a5-f461-41bc-b53a-b3622a09d6af admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:25.866555 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542305d-bf68-48b1-b299-57f72bfba960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:25.867318 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542305d-bf68-48b1-b299-57f72bfba960 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:25.867670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542305d-bf68-48b1-b299-57f72bfba960 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:25.867999 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542305d-bf68-48b1-b299-57f72bfba960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:25.877491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542305d-bf68-48b1-b299-57f72bfba960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:25.878961 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542305d-bf68-48b1-b299-57f72bfba960 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:25.889774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 969/1949] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:26.329043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462afcbb-b719-4433-a8db-3e19658ad358 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:26.332289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-462afcbb-b719-4433-a8db-3e19658ad358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:26.346828 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462afcbb-b719-4433-a8db-3e19658ad358 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:26.353267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:26.354434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:26.354922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:26.355831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] PATH_INFO: `/v3/projects/191d26a45a4a4aa5b0847a717c8007f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:26.358238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=191d26a45a4a4aa5b0847a717c8007f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:26.360931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:26.416968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '191d26a45a4a4aa5b0847a717c8007f3', 'request_id': 'req-9108dcde-4de8-447e-b933-41781c9364b2'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:26.426017 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '191d26a45a4a4aa5b0847a717c8007f3', 'request_id': 'req-9108dcde-4de8-447e-b933-41781c9364b2'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:26.438003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '191d26a45a4a4aa5b0847a717c8007f3', 'request_id': 'req-9108dcde-4de8-447e-b933-41781c9364b2'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:26.453890 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] The token cache is being invalidate because project 191d26a45a4a4aa5b0847a717c8007f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:26.454287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:26.455004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:26.456690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 981/1950] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:26 2022] DELETE /identity/v3/projects/191d26a45a4a4aa5b0847a717c8007f3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:26.461795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542305d-bf68-48b1-b299-57f72bfba960 service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:26.481857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542305d-bf68-48b1-b299-57f72bfba960 service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:26.546119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80039ba7-82e8-4392-9961-53cbe3211b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:26.547074 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80039ba7-82e8-4392-9961-53cbe3211b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:26.547363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80039ba7-82e8-4392-9961-53cbe3211b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:26.547636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80039ba7-82e8-4392-9961-53cbe3211b48 admin admin] PATH_INFO: `/v3/users/9d45a430b3a4425ca7773c5da77b8b6a` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:26.549192 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80039ba7-82e8-4392-9961-53cbe3211b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d45a430b3a4425ca7773c5da77b8b6a)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:26.551879 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80039ba7-82e8-4392-9961-53cbe3211b48 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:26.633056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-80039ba7-82e8-4392-9961-53cbe3211b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d45a430b3a4425ca7773c5da77b8b6a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:26.634736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-80039ba7-82e8-4392-9961-53cbe3211b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d45a430b3a4425ca7773c5da77b8b6a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:26.636860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-80039ba7-82e8-4392-9961-53cbe3211b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d45a430b3a4425ca7773c5da77b8b6a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:26.657547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-80039ba7-82e8-4392-9961-53cbe3211b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d45a430b3a4425ca7773c5da77b8b6a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:26.669779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-80039ba7-82e8-4392-9961-53cbe3211b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d45a430b3a4425ca7773c5da77b8b6a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:26.695968 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 970/1951] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:26 2022] DELETE /identity/v3/users/9d45a430b3a4425ca7773c5da77b8b6a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:27.096067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:27.098248 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:27.111928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108dcde-4de8-447e-b933-41781c9364b2 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:27.205065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:27.206212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:27.206500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:27.206777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f admin admin] PATH_INFO: `/v3/projects/89c1466b7a194c91bcd166f02d33a822` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:27.207993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=89c1466b7a194c91bcd166f02d33a822)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:27.212030 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:27.307031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89c1466b7a194c91bcd166f02d33a822', 'request_id': 'req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:27.314121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89c1466b7a194c91bcd166f02d33a822', 'request_id': 'req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:27.325391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89c1466b7a194c91bcd166f02d33a822', 'request_id': 'req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:27.358339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f admin admin] The token cache is being invalidate because project 89c1466b7a194c91bcd166f02d33a822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:27.358849 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:27.360240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fde16dbf-e159-4e2b-bbfb-14dc79a18d4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:27.365346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 982/1952] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:27 2022] DELETE /identity/v3/projects/89c1466b7a194c91bcd166f02d33a822 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:27.371753 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1858b-69d1-4234-a176-62f68eba9b43 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:27.372964 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1858b-69d1-4234-a176-62f68eba9b43 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:27.374599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1858b-69d1-4234-a176-62f68eba9b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:27.408932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-74f1858b-69d1-4234-a176-62f68eba9b43 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:27.411771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-74f1858b-69d1-4234-a176-62f68eba9b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:27.526169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 971/1953] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:27.534187 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13906ff1-ce70-4a41-871a-c1bbe4480336 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:27.534478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13906ff1-ce70-4a41-871a-c1bbe4480336 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:27.534760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13906ff1-ce70-4a41-871a-c1bbe4480336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:27.569453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-13906ff1-ce70-4a41-871a-c1bbe4480336 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:27.570655 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-13906ff1-ce70-4a41-871a-c1bbe4480336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:27.599018 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 983/1954] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:27.603099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1858b-69d1-4234-a176-62f68eba9b43 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:27.616558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1858b-69d1-4234-a176-62f68eba9b43 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:27.638212 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeff710-e41c-4034-9928-0f8ba58629c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:27.638908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeff710-e41c-4034-9928-0f8ba58629c0 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:27.639182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeff710-e41c-4034-9928-0f8ba58629c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:27.639612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeff710-e41c-4034-9928-0f8ba58629c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:27.640514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeff710-e41c-4034-9928-0f8ba58629c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:27.640988 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeff710-e41c-4034-9928-0f8ba58629c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:27.645467 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeff710-e41c-4034-9928-0f8ba58629c0 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:27.671286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 972/1955] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:27:27.675869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13906ff1-ce70-4a41-871a-c1bbe4480336 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:27.687084 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13906ff1-ce70-4a41-871a-c1bbe4480336 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:27.693053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968911ca-4bb4-4cf1-aa35-00b5118d0544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:27.694298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968911ca-4bb4-4cf1-aa35-00b5118d0544 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:27.695020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968911ca-4bb4-4cf1-aa35-00b5118d0544 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:27.695501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968911ca-4bb4-4cf1-aa35-00b5118d0544 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:27.696645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968911ca-4bb4-4cf1-aa35-00b5118d0544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:27.699054 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968911ca-4bb4-4cf1-aa35-00b5118d0544 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:27.747850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 984/1956] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:27 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:27.752136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeff710-e41c-4034-9928-0f8ba58629c0 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:27.768859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeff710-e41c-4034-9928-0f8ba58629c0 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:27.848926 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880053f-514d-43de-91b9-a6ddfb28ae07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:27.851019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880053f-514d-43de-91b9-a6ddfb28ae07 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:27.851518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880053f-514d-43de-91b9-a6ddfb28ae07 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:27.851979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880053f-514d-43de-91b9-a6ddfb28ae07 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:27.854167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880053f-514d-43de-91b9-a6ddfb28ae07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:27.860079 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880053f-514d-43de-91b9-a6ddfb28ae07 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:27.906950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 973/1957] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:27 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:27:27.910672 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968911ca-4bb4-4cf1-aa35-00b5118d0544 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:27.931046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968911ca-4bb4-4cf1-aa35-00b5118d0544 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:27.962227 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32576d4f-1931-4e11-bd67-e8205a75b071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:27.966516 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32576d4f-1931-4e11-bd67-e8205a75b071 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:27.967836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32576d4f-1931-4e11-bd67-e8205a75b071 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:27.968295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32576d4f-1931-4e11-bd67-e8205a75b071 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:27.970710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32576d4f-1931-4e11-bd67-e8205a75b071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:27.980717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32576d4f-1931-4e11-bd67-e8205a75b071 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:28.026020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 985/1958] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:28.030204 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880053f-514d-43de-91b9-a6ddfb28ae07 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:28.048349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880053f-514d-43de-91b9-a6ddfb28ae07 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:28.067054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07364ba-a0e1-4f14-85b1-f599911fb605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:28.068084 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07364ba-a0e1-4f14-85b1-f599911fb605 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:28.068572 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07364ba-a0e1-4f14-85b1-f599911fb605 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:28.069027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07364ba-a0e1-4f14-85b1-f599911fb605 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:28.070002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07364ba-a0e1-4f14-85b1-f599911fb605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:28.071949 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07364ba-a0e1-4f14-85b1-f599911fb605 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:28.093043 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 974/1959] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:28.099063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32576d4f-1931-4e11-bd67-e8205a75b071 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:28.133877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32576d4f-1931-4e11-bd67-e8205a75b071 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:28.150288 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8f6e3-8729-4e41-98c3-1525caf31927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:28.151957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f6e3-8729-4e41-98c3-1525caf31927 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:28.152445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f6e3-8729-4e41-98c3-1525caf31927 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:28.152935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f6e3-8729-4e41-98c3-1525caf31927 admin admin] PATH_INFO: `/v3/projects/1320e5ca93b24bcb964c2e8378a8b44c/users/c9f0d867830d491f95777368ae3fc904/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:28.203156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8f6e3-8729-4e41-98c3-1525caf31927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1320e5ca93b24bcb964c2e8378a8b44c, user_id=c9f0d867830d491f95777368ae3fc904, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:28.206009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8f6e3-8729-4e41-98c3-1525caf31927 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:28.232543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 986/1960] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:28 2022] PUT /identity/v3/projects/1320e5ca93b24bcb964c2e8378a8b44c/users/c9f0d867830d491f95777368ae3fc904/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:28.243349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d8642-10b7-4127-8bae-857121c0679d None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:28.243636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d8642-10b7-4127-8bae-857121c0679d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:28.244313 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d8642-10b7-4127-8bae-857121c0679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:28.283070 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-2b2d8642-10b7-4127-8bae-857121c0679d None None] MFA Rules not processed for user `c9f0d867830d491f95777368ae3fc904`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:28.284930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-2b2d8642-10b7-4127-8bae-857121c0679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:28.383526 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 975/1961] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:28.385668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8f6e3-8729-4e41-98c3-1525caf31927 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:28.462912 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8f6e3-8729-4e41-98c3-1525caf31927 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:28.468299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd876c-ebdf-4e35-bbf7-dd8e64b9b619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:28.469303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd876c-ebdf-4e35-bbf7-dd8e64b9b619 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:28.469585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd876c-ebdf-4e35-bbf7-dd8e64b9b619 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:28.469858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd876c-ebdf-4e35-bbf7-dd8e64b9b619 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:28.470567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd876c-ebdf-4e35-bbf7-dd8e64b9b619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:28.472536 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd876c-ebdf-4e35-bbf7-dd8e64b9b619 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:28.512169 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 987/1962] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:28 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:28.516889 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d8642-10b7-4127-8bae-857121c0679d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:28.531048 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d8642-10b7-4127-8bae-857121c0679d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:28.604197 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fa687-f7e2-450b-87de-f24224a27304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:28.604897 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fa687-f7e2-450b-87de-f24224a27304 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:28.605547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fa687-f7e2-450b-87de-f24224a27304 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:28.606029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133fa687-f7e2-450b-87de-f24224a27304 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:28.606888 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133fa687-f7e2-450b-87de-f24224a27304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:28.608981 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133fa687-f7e2-450b-87de-f24224a27304 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:28.641853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 976/1963] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:28 2022] POST /identity/v3/users => generated 300 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:28.643743 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd876c-ebdf-4e35-bbf7-dd8e64b9b619 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:28.655434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd876c-ebdf-4e35-bbf7-dd8e64b9b619 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:28.660843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62add6-ca69-40df-a6e2-8f8210a01f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:28.661589 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62add6-ca69-40df-a6e2-8f8210a01f90 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:28.661939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62add6-ca69-40df-a6e2-8f8210a01f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:28.662311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62add6-ca69-40df-a6e2-8f8210a01f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:28.663348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62add6-ca69-40df-a6e2-8f8210a01f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:28.665013 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62add6-ca69-40df-a6e2-8f8210a01f90 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:28.675600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 988/1964] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:28.680809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fa687-f7e2-450b-87de-f24224a27304 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:28.691770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133fa687-f7e2-450b-87de-f24224a27304 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:28.698855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68981f8-ed9d-4513-a656-21732291567a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:28.699753 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68981f8-ed9d-4513-a656-21732291567a admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:28.700076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68981f8-ed9d-4513-a656-21732291567a admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:28.700352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68981f8-ed9d-4513-a656-21732291567a admin admin] PATH_INFO: `/v3/projects/148917291d14405e99c7228b4f04f39b/users/ebc25a64e1d34550a313f2ebae7c88bb/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:28.717284 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68981f8-ed9d-4513-a656-21732291567a admin admin] RBAC: Authorizing `identity:create_grant(project_id=148917291d14405e99c7228b4f04f39b, user_id=ebc25a64e1d34550a313f2ebae7c88bb, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:28.720352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68981f8-ed9d-4513-a656-21732291567a admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:28.734383 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 977/1965] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:28 2022] PUT /identity/v3/projects/148917291d14405e99c7228b4f04f39b/users/ebc25a64e1d34550a313f2ebae7c88bb/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:28.738644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62add6-ca69-40df-a6e2-8f8210a01f90 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:28.755046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62add6-ca69-40df-a6e2-8f8210a01f90 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:28.818078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0256fb-7e8f-4373-8c24-2f8cdda6e50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:28.818949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0256fb-7e8f-4373-8c24-2f8cdda6e50b admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:28.819296 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0256fb-7e8f-4373-8c24-2f8cdda6e50b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:28.819604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0256fb-7e8f-4373-8c24-2f8cdda6e50b admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:28.820750 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0256fb-7e8f-4373-8c24-2f8cdda6e50b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:28.822388 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0256fb-7e8f-4373-8c24-2f8cdda6e50b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:28.833502 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 989/1966] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:28.838372 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68981f8-ed9d-4513-a656-21732291567a admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:28.848306 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68981f8-ed9d-4513-a656-21732291567a admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:28.853605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312490f-775a-42ef-aa71-dc83d8770420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:28.854483 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312490f-775a-42ef-aa71-dc83d8770420 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:28.854765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312490f-775a-42ef-aa71-dc83d8770420 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:28.855138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312490f-775a-42ef-aa71-dc83d8770420 admin admin] PATH_INFO: `/v3/projects/148917291d14405e99c7228b4f04f39b/users/ebc25a64e1d34550a313f2ebae7c88bb/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:28.857250 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312490f-775a-42ef-aa71-dc83d8770420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=148917291d14405e99c7228b4f04f39b, user_id=ebc25a64e1d34550a313f2ebae7c88bb, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:28.859918 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312490f-775a-42ef-aa71-dc83d8770420 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:28.870245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 978/1967] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:28 2022] PUT /identity/v3/projects/148917291d14405e99c7228b4f04f39b/users/ebc25a64e1d34550a313f2ebae7c88bb/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:28.881753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ca037-b1da-48b7-96e1-569c95f6c2c3 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:28.882110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ca037-b1da-48b7-96e1-569c95f6c2c3 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:28.882400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ca037-b1da-48b7-96e1-569c95f6c2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:28.911200 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-945ca037-b1da-48b7-96e1-569c95f6c2c3 None None] MFA Rules not processed for user `ebc25a64e1d34550a313f2ebae7c88bb`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:28.913010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-945ca037-b1da-48b7-96e1-569c95f6c2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:28.989746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 990/1968] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:28.998930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312490f-775a-42ef-aa71-dc83d8770420 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:29.000778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c312490f-775a-42ef-aa71-dc83d8770420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:29.064244 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312490f-775a-42ef-aa71-dc83d8770420 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:29.067871 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40415e-5a4b-47bc-aaa6-a5b9bbfb035a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:29.068558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40415e-5a4b-47bc-aaa6-a5b9bbfb035a service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:29.068849 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40415e-5a4b-47bc-aaa6-a5b9bbfb035a service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:29.069194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40415e-5a4b-47bc-aaa6-a5b9bbfb035a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:29.078681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40415e-5a4b-47bc-aaa6-a5b9bbfb035a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:29.080135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40415e-5a4b-47bc-aaa6-a5b9bbfb035a service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:29.092170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 979/1969] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:29.112108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ca037-b1da-48b7-96e1-569c95f6c2c3 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:29.114126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-945ca037-b1da-48b7-96e1-569c95f6c2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:29.191768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ca037-b1da-48b7-96e1-569c95f6c2c3 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:29.198193 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981afd62-0072-4d2f-b0ce-ccca16d1a1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:29.200333 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981afd62-0072-4d2f-b0ce-ccca16d1a1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:29.200711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981afd62-0072-4d2f-b0ce-ccca16d1a1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:29.201028 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981afd62-0072-4d2f-b0ce-ccca16d1a1f7 admin admin] PATH_INFO: `/v3/users/2346ed4941fa45a3bb0578bde8581f27` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:29.202489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981afd62-0072-4d2f-b0ce-ccca16d1a1f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2346ed4941fa45a3bb0578bde8581f27)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:29.207650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981afd62-0072-4d2f-b0ce-ccca16d1a1f7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:29.251054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40415e-5a4b-47bc-aaa6-a5b9bbfb035a service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:29.263695 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40415e-5a4b-47bc-aaa6-a5b9bbfb035a service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:29.268742 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6045f6-869b-47d9-9d0a-a5e3503b3cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:29.269602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6045f6-869b-47d9-9d0a-a5e3503b3cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:29.269880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6045f6-869b-47d9-9d0a-a5e3503b3cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:29.270157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6045f6-869b-47d9-9d0a-a5e3503b3cdf admin admin] PATH_INFO: `/v3/users/c9f0d867830d491f95777368ae3fc904` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:29.271411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6045f6-869b-47d9-9d0a-a5e3503b3cdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f0d867830d491f95777368ae3fc904)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:29.274302 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6045f6-869b-47d9-9d0a-a5e3503b3cdf admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:29.276770 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-981afd62-0072-4d2f-b0ce-ccca16d1a1f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2346ed4941fa45a3bb0578bde8581f27'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.277391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-981afd62-0072-4d2f-b0ce-ccca16d1a1f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2346ed4941fa45a3bb0578bde8581f27'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.288276 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-981afd62-0072-4d2f-b0ce-ccca16d1a1f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2346ed4941fa45a3bb0578bde8581f27'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.300736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-981afd62-0072-4d2f-b0ce-ccca16d1a1f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2346ed4941fa45a3bb0578bde8581f27'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.301640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-981afd62-0072-4d2f-b0ce-ccca16d1a1f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2346ed4941fa45a3bb0578bde8581f27'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.323139 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 991/1970] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:29 2022] DELETE /identity/v3/users/2346ed4941fa45a3bb0578bde8581f27 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:29.327501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf60820-7d29-4c55-8988-b158b922608b None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:29.327789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf60820-7d29-4c55-8988-b158b922608b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:29.328145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3c6045f6-869b-47d9-9d0a-a5e3503b3cdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f0d867830d491f95777368ae3fc904'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.328328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf60820-7d29-4c55-8988-b158b922608b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:29.328808 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3c6045f6-869b-47d9-9d0a-a5e3503b3cdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f0d867830d491f95777368ae3fc904'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.329433 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3c6045f6-869b-47d9-9d0a-a5e3503b3cdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f0d867830d491f95777368ae3fc904'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.337542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3c6045f6-869b-47d9-9d0a-a5e3503b3cdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f0d867830d491f95777368ae3fc904'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.344868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3c6045f6-869b-47d9-9d0a-a5e3503b3cdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f0d867830d491f95777368ae3fc904'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.346236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2bf60820-7d29-4c55-8988-b158b922608b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:29.347575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2bf60820-7d29-4c55-8988-b158b922608b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:29.370023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 980/1971] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:29 2022] DELETE /identity/v3/users/c9f0d867830d491f95777368ae3fc904 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:29.373754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84528-cc7b-46f3-8ae7-d1b06efc9c7b None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:29.374054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84528-cc7b-46f3-8ae7-d1b06efc9c7b None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:29.374757 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84528-cc7b-46f3-8ae7-d1b06efc9c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:29.403251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d1b84528-cc7b-46f3-8ae7-d1b06efc9c7b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:29.405621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d1b84528-cc7b-46f3-8ae7-d1b06efc9c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:29.461642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 992/1972] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:29.468789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf60820-7d29-4c55-8988-b158b922608b None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:29.470214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2bf60820-7d29-4c55-8988-b158b922608b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:29.505940 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 981/1973] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:29.516895 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b84528-cc7b-46f3-8ae7-d1b06efc9c7b None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:29.518735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d1b84528-cc7b-46f3-8ae7-d1b06efc9c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:29.583095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf60820-7d29-4c55-8988-b158b922608b None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:29.646508 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b84528-cc7b-46f3-8ae7-d1b06efc9c7b None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:29.653495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94705e4-70d4-4c0a-a938-5e11b101c153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:29.654470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94705e4-70d4-4c0a-a938-5e11b101c153 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:29.654932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da40fc3-3c31-4fba-9cbc-f1b9f0dd1296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:29.655077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94705e4-70d4-4c0a-a938-5e11b101c153 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:29.655480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94705e4-70d4-4c0a-a938-5e11b101c153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:29.655592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da40fc3-3c31-4fba-9cbc-f1b9f0dd1296 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:29.655875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da40fc3-3c31-4fba-9cbc-f1b9f0dd1296 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:29.656206 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da40fc3-3c31-4fba-9cbc-f1b9f0dd1296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:29.668117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94705e4-70d4-4c0a-a938-5e11b101c153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:29.670741 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94705e4-70d4-4c0a-a938-5e11b101c153 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:29.675458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da40fc3-3c31-4fba-9cbc-f1b9f0dd1296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:29.677790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da40fc3-3c31-4fba-9cbc-f1b9f0dd1296 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:29.685136 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 993/1974] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:27:29.686016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94705e4-70d4-4c0a-a938-5e11b101c153 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:29.687459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c94705e4-70d4-4c0a-a938-5e11b101c153 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:29.703220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94705e4-70d4-4c0a-a938-5e11b101c153 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:29.707488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 982/1975] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:27:29.727125 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:29.728639 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:29.728920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:29.729811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] PATH_INFO: `/v3/users/609e17667a2a4c519170cb96bd50a749` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:29.731197 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=609e17667a2a4c519170cb96bd50a749)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:29.733961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:29.865438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '609e17667a2a4c519170cb96bd50a749'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.867016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '609e17667a2a4c519170cb96bd50a749'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.890610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '609e17667a2a4c519170cb96bd50a749'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.904820 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '609e17667a2a4c519170cb96bd50a749'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.905883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '609e17667a2a4c519170cb96bd50a749'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:29.947329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 994/1976] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:29 2022] DELETE /identity/v3/users/609e17667a2a4c519170cb96bd50a749 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:29.951971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb80a8-b787-426f-bfff-4d14fdac355c None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:29.952437 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb80a8-b787-426f-bfff-4d14fdac355c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:29.952878 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb80a8-b787-426f-bfff-4d14fdac355c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:29.986156 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-5ebb80a8-b787-426f-bfff-4d14fdac355c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:29.988179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5ebb80a8-b787-426f-bfff-4d14fdac355c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:30.143813 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 983/1977] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:30.186387 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:30.188488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:30.283464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b958b-399b-4349-873a-98646fa1e5ab admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:30.288330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63adbde-b3ee-466a-89fe-108e2b6af588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:30.288995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63adbde-b3ee-466a-89fe-108e2b6af588 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:30.289282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63adbde-b3ee-466a-89fe-108e2b6af588 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:30.289729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63adbde-b3ee-466a-89fe-108e2b6af588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:30.306859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63adbde-b3ee-466a-89fe-108e2b6af588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:30.311014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63adbde-b3ee-466a-89fe-108e2b6af588 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:30.328377 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 995/1978] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:30.462106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb80a8-b787-426f-bfff-4d14fdac355c None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:30.464401 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5ebb80a8-b787-426f-bfff-4d14fdac355c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:30.473637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63adbde-b3ee-466a-89fe-108e2b6af588 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:30.476366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f63adbde-b3ee-466a-89fe-108e2b6af588 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:30.491771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb80a8-b787-426f-bfff-4d14fdac355c None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:30.496574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63adbde-b3ee-466a-89fe-108e2b6af588 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:30.502586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:30.503913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:30.503913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:30.504213 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] PATH_INFO: `/v3/projects/d209ebca9c7a452bab3305415aec7116` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:30.509612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d209ebca9c7a452bab3305415aec7116)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:30.511631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:30.518685 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:30.519528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:30.519812 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:30.520114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] PATH_INFO: `/v3/projects/1320e5ca93b24bcb964c2e8378a8b44c` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:30.521878 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1320e5ca93b24bcb964c2e8378a8b44c)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:30.524024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:30.617310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd209ebca9c7a452bab3305415aec7116', 'request_id': 'req-667253cd-65b3-4b7b-9c2d-8de79df4faff'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.625245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd209ebca9c7a452bab3305415aec7116', 'request_id': 'req-667253cd-65b3-4b7b-9c2d-8de79df4faff'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.631978 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd209ebca9c7a452bab3305415aec7116', 'request_id': 'req-667253cd-65b3-4b7b-9c2d-8de79df4faff'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.637721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1320e5ca93b24bcb964c2e8378a8b44c', 'request_id': 'req-aa08b436-5eb3-41d1-9b51-439738b07024'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.644832 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1320e5ca93b24bcb964c2e8378a8b44c', 'request_id': 'req-aa08b436-5eb3-41d1-9b51-439738b07024'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.649924 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] The token cache is being invalidate because project d209ebca9c7a452bab3305415aec7116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:30.651131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.652416 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.654405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 984/1979] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:30 2022] DELETE /identity/v3/projects/d209ebca9c7a452bab3305415aec7116 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:30.658347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:30.658837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1320e5ca93b24bcb964c2e8378a8b44c', 'request_id': 'req-aa08b436-5eb3-41d1-9b51-439738b07024'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.661081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:30.692306 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] The token cache is being invalidate because project 1320e5ca93b24bcb964c2e8378a8b44c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:30.693276 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.694349 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.698597 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 996/1980] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:30 2022] DELETE /identity/v3/projects/1320e5ca93b24bcb964c2e8378a8b44c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:27:30.702870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:30.705429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:30.775739 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667253cd-65b3-4b7b-9c2d-8de79df4faff admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:30.800257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08b436-5eb3-41d1-9b51-439738b07024 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:30.816060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:30.817543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:30.818294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:30.819019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] PATH_INFO: `/v3/users/ebc25a64e1d34550a313f2ebae7c88bb` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:30.820441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebc25a64e1d34550a313f2ebae7c88bb)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:30.822774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:30.858489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727298d-c424-4d1c-900d-dd010d95d037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:30.859520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727298d-c424-4d1c-900d-dd010d95d037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:30.859803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727298d-c424-4d1c-900d-dd010d95d037 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:30.860108 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727298d-c424-4d1c-900d-dd010d95d037 admin admin] PATH_INFO: `/v3/users/d1d7996cd04d442395e910e81ca71993` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:30.861320 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727298d-c424-4d1c-900d-dd010d95d037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d7996cd04d442395e910e81ca71993)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:30.863469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727298d-c424-4d1c-900d-dd010d95d037 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:30.891522 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebc25a64e1d34550a313f2ebae7c88bb'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.892371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebc25a64e1d34550a313f2ebae7c88bb'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.902915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebc25a64e1d34550a313f2ebae7c88bb'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.910483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebc25a64e1d34550a313f2ebae7c88bb'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.911174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebc25a64e1d34550a313f2ebae7c88bb'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.915988 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f727298d-c424-4d1c-900d-dd010d95d037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d7996cd04d442395e910e81ca71993'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.916658 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f727298d-c424-4d1c-900d-dd010d95d037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d7996cd04d442395e910e81ca71993'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.918034 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f727298d-c424-4d1c-900d-dd010d95d037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d7996cd04d442395e910e81ca71993'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.925330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f727298d-c424-4d1c-900d-dd010d95d037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d7996cd04d442395e910e81ca71993'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.930773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 997/1981] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:30 2022] DELETE /identity/v3/users/ebc25a64e1d34550a313f2ebae7c88bb => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:30.931606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:30.933003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:30.933207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f727298d-c424-4d1c-900d-dd010d95d037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d7996cd04d442395e910e81ca71993'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:30.956551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 985/1982] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:30 2022] DELETE /identity/v3/users/d1d7996cd04d442395e910e81ca71993 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:31.025992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d508c-ab51-403c-862a-ab4968beccdb admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:31.086935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31659a4c-cc98-46d0-8f57-e9d7da5bebce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:31.087916 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31659a4c-cc98-46d0-8f57-e9d7da5bebce admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.088531 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31659a4c-cc98-46d0-8f57-e9d7da5bebce admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.088531 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31659a4c-cc98-46d0-8f57-e9d7da5bebce admin admin] PATH_INFO: `/v3/projects/d0e4adac41384d92b92e3011554cca8b` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.090020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31659a4c-cc98-46d0-8f57-e9d7da5bebce admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0e4adac41384d92b92e3011554cca8b)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:31.091629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31659a4c-cc98-46d0-8f57-e9d7da5bebce admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:31.153039 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-31659a4c-cc98-46d0-8f57-e9d7da5bebce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0e4adac41384d92b92e3011554cca8b', 'request_id': 'req-31659a4c-cc98-46d0-8f57-e9d7da5bebce'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.164624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-31659a4c-cc98-46d0-8f57-e9d7da5bebce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0e4adac41384d92b92e3011554cca8b', 'request_id': 'req-31659a4c-cc98-46d0-8f57-e9d7da5bebce'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.175617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-31659a4c-cc98-46d0-8f57-e9d7da5bebce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0e4adac41384d92b92e3011554cca8b', 'request_id': 'req-31659a4c-cc98-46d0-8f57-e9d7da5bebce'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.196437 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-31659a4c-cc98-46d0-8f57-e9d7da5bebce admin admin] The token cache is being invalidate because project d0e4adac41384d92b92e3011554cca8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:31.197042 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-31659a4c-cc98-46d0-8f57-e9d7da5bebce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.197629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-31659a4c-cc98-46d0-8f57-e9d7da5bebce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.199349 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 998/1983] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:30 2022] DELETE /identity/v3/projects/d0e4adac41384d92b92e3011554cca8b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:31.204848 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb41d4e-ba38-4fff-889a-4eafeefad6b9 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.205247 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb41d4e-ba38-4fff-889a-4eafeefad6b9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.205593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb41d4e-ba38-4fff-889a-4eafeefad6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.227534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-3fb41d4e-ba38-4fff-889a-4eafeefad6b9 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:31.229435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3fb41d4e-ba38-4fff-889a-4eafeefad6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:31.315913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 986/1984] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:31.317053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac1d2a-d4ab-4f97-9b55-63eeb310c124 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.317364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac1d2a-d4ab-4f97-9b55-63eeb310c124 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.317643 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac1d2a-d4ab-4f97-9b55-63eeb310c124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.342174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-0aac1d2a-d4ab-4f97-9b55-63eeb310c124 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:31.343830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-0aac1d2a-d4ab-4f97-9b55-63eeb310c124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:31.357329 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb41d4e-ba38-4fff-889a-4eafeefad6b9 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:31.358839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3fb41d4e-ba38-4fff-889a-4eafeefad6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:31.362809 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 999/1985] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:31.363993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac1d2a-d4ab-4f97-9b55-63eeb310c124 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:31.385162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb41d4e-ba38-4fff-889a-4eafeefad6b9 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:31.385321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac1d2a-d4ab-4f97-9b55-63eeb310c124 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:31.392143 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c846e-aead-4afe-80d5-92f08d291fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:31.392926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c846e-aead-4afe-80d5-92f08d291fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.393206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c846e-aead-4afe-80d5-92f08d291fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.393763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c846e-aead-4afe-80d5-92f08d291fd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.394556 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c846e-aead-4afe-80d5-92f08d291fd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:31.394913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c846e-aead-4afe-80d5-92f08d291fd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:31.403202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:31.404230 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.404448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.404727 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] PATH_INFO: `/v3/projects/84c3842cf96d4fc9925533012385c381` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.406087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84c3842cf96d4fc9925533012385c381)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:31.409629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:31.413592 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c846e-aead-4afe-80d5-92f08d291fd0 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:31.430279 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1000/1986] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:27:31.431261 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c846e-aead-4afe-80d5-92f08d291fd0 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:31.433677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-0b2c846e-aead-4afe-80d5-92f08d291fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:31.457352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c846e-aead-4afe-80d5-92f08d291fd0 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:31.471430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72117825-421f-4a44-98fd-bc5281127840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:31.473149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72117825-421f-4a44-98fd-bc5281127840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.473430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72117825-421f-4a44-98fd-bc5281127840 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.473744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72117825-421f-4a44-98fd-bc5281127840 admin admin] PATH_INFO: `/v3/projects/148917291d14405e99c7228b4f04f39b` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.482824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72117825-421f-4a44-98fd-bc5281127840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=148917291d14405e99c7228b4f04f39b)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:31.484798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72117825-421f-4a44-98fd-bc5281127840 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:31.525104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84c3842cf96d4fc9925533012385c381', 'request_id': 'req-f1b4a674-228e-4162-87fd-248928ab0ea3'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.532462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84c3842cf96d4fc9925533012385c381', 'request_id': 'req-f1b4a674-228e-4162-87fd-248928ab0ea3'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.540515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84c3842cf96d4fc9925533012385c381', 'request_id': 'req-f1b4a674-228e-4162-87fd-248928ab0ea3'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.554774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-72117825-421f-4a44-98fd-bc5281127840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '148917291d14405e99c7228b4f04f39b', 'request_id': 'req-72117825-421f-4a44-98fd-bc5281127840'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.560201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] The token cache is being invalidate because project 84c3842cf96d4fc9925533012385c381 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:31.560494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.561166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.564171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 987/1987] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:31 2022] DELETE /identity/v3/projects/84c3842cf96d4fc9925533012385c381 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:31.565171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:31.565699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-72117825-421f-4a44-98fd-bc5281127840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '148917291d14405e99c7228b4f04f39b', 'request_id': 'req-72117825-421f-4a44-98fd-bc5281127840'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.567024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:31.579927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-72117825-421f-4a44-98fd-bc5281127840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '148917291d14405e99c7228b4f04f39b', 'request_id': 'req-72117825-421f-4a44-98fd-bc5281127840'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.594258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-72117825-421f-4a44-98fd-bc5281127840 admin admin] The token cache is being invalidate because project 148917291d14405e99c7228b4f04f39b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:31.594692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-72117825-421f-4a44-98fd-bc5281127840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.595617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-72117825-421f-4a44-98fd-bc5281127840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:31.597850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1001/1988] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:31 2022] DELETE /identity/v3/projects/148917291d14405e99c7228b4f04f39b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:31.602321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5939f22-6407-4980-8bb0-39c4d99faf54 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.602852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5939f22-6407-4980-8bb0-39c4d99faf54 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.603008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5939f22-6407-4980-8bb0-39c4d99faf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.640968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-d5939f22-6407-4980-8bb0-39c4d99faf54 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:31.642283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d5939f22-6407-4980-8bb0-39c4d99faf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:31.664182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4a674-228e-4162-87fd-248928ab0ea3 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:31.760697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1002/1989] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:31.760697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d604580-73af-4edd-afe7-8ceab285dd76 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.760697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d604580-73af-4edd-afe7-8ceab285dd76 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.760697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d604580-73af-4edd-afe7-8ceab285dd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.793366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4ccf6-f61b-4714-84fe-8850f33a87dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:31.794103 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4ccf6-f61b-4714-84fe-8850f33a87dc admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.794492 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4ccf6-f61b-4714-84fe-8850f33a87dc admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.794779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4ccf6-f61b-4714-84fe-8850f33a87dc admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.795504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4ccf6-f61b-4714-84fe-8850f33a87dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:31.796931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-4d604580-73af-4edd-afe7-8ceab285dd76 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:31.797858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4ccf6-f61b-4714-84fe-8850f33a87dc admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:31.798169 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4d604580-73af-4edd-afe7-8ceab285dd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:31.811003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1003/1990] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:31.812646 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4dd25-73c8-49e2-a77a-8e6644dd6f7f None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.812966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4dd25-73c8-49e2-a77a-8e6644dd6f7f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.813220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4dd25-73c8-49e2-a77a-8e6644dd6f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.830510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 988/1991] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:31 2022] POST /identity/v3/projects => generated 373 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:31.832062 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7241e-512f-4e43-bf66-9b2a91b8036e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.832603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7241e-512f-4e43-bf66-9b2a91b8036e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.833309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7241e-512f-4e43-bf66-9b2a91b8036e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.839331 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-31f4dd25-73c8-49e2-a77a-8e6644dd6f7f None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:31.840641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-31f4dd25-73c8-49e2-a77a-8e6644dd6f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:31.857379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-00a7241e-512f-4e43-bf66-9b2a91b8036e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:31.858694 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-00a7241e-512f-4e43-bf66-9b2a91b8036e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:31.910238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1004/1992] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:31.910238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4dd25-73c8-49e2-a77a-8e6644dd6f7f None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:31.910587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-31f4dd25-73c8-49e2-a77a-8e6644dd6f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:31.922118 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4dd25-73c8-49e2-a77a-8e6644dd6f7f None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:31.924753 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 989/1993] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:31.925876 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7241e-512f-4e43-bf66-9b2a91b8036e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:31.927235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc496c-cd0a-405c-9c7b-28ea0f277859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:31.929063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc496c-cd0a-405c-9c7b-28ea0f277859 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.929629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc496c-cd0a-405c-9c7b-28ea0f277859 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.929913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc496c-cd0a-405c-9c7b-28ea0f277859 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.930621 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc496c-cd0a-405c-9c7b-28ea0f277859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:31.933549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc496c-cd0a-405c-9c7b-28ea0f277859 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:31.936209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7241e-512f-4e43-bf66-9b2a91b8036e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:31.940828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79582ce-edf5-40ed-923c-95c2aa46afea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:31.941684 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79582ce-edf5-40ed-923c-95c2aa46afea admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.941967 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79582ce-edf5-40ed-923c-95c2aa46afea admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.942693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79582ce-edf5-40ed-923c-95c2aa46afea admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.943423 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79582ce-edf5-40ed-923c-95c2aa46afea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:31.944526 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79582ce-edf5-40ed-923c-95c2aa46afea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:31.946211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79582ce-edf5-40ed-923c-95c2aa46afea admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:31.957178 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1005/1994] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:31 2022] POST /identity/v3/users => generated 287 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:31.958032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc496c-cd0a-405c-9c7b-28ea0f277859 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:31.966167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 990/1995] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:27:31.966167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79582ce-edf5-40ed-923c-95c2aa46afea admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:31.970731 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc496c-cd0a-405c-9c7b-28ea0f277859 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:31.976183 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed813ce2-b645-45a3-aaeb-a4b1b186bbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:31.976852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed813ce2-b645-45a3-aaeb-a4b1b186bbdf admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.977141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed813ce2-b645-45a3-aaeb-a4b1b186bbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.977404 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed813ce2-b645-45a3-aaeb-a4b1b186bbdf admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.977809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79582ce-edf5-40ed-923c-95c2aa46afea admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:31.978129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed813ce2-b645-45a3-aaeb-a4b1b186bbdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:31.978464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed813ce2-b645-45a3-aaeb-a4b1b186bbdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:31.980212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed813ce2-b645-45a3-aaeb-a4b1b186bbdf admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:31.982870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e905028-6466-4c5a-89df-134c2af09c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:31.983752 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e905028-6466-4c5a-89df-134c2af09c5e admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:31.983971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e905028-6466-4c5a-89df-134c2af09c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:31.984283 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e905028-6466-4c5a-89df-134c2af09c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:31.985016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e905028-6466-4c5a-89df-134c2af09c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:31.986591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e905028-6466-4c5a-89df-134c2af09c5e admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:31.997510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1006/1996] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:27:31.998174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed813ce2-b645-45a3-aaeb-a4b1b186bbdf admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.001225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 991/1997] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:32.001999 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e905028-6466-4c5a-89df-134c2af09c5e admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.009076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed813ce2-b645-45a3-aaeb-a4b1b186bbdf admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.012492 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e905028-6466-4c5a-89df-134c2af09c5e admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.015876 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03628330-9377-4cfc-98d1-532add31e9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.016686 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03628330-9377-4cfc-98d1-532add31e9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.017080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03628330-9377-4cfc-98d1-532add31e9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.017449 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03628330-9377-4cfc-98d1-532add31e9c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.017720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcdf317-1286-4ee4-a609-567463f51926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.018285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03628330-9377-4cfc-98d1-532add31e9c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.018398 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdf317-1286-4ee4-a609-567463f51926 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.018643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdf317-1286-4ee4-a609-567463f51926 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.019283 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdf317-1286-4ee4-a609-567463f51926 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.019708 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcdf317-1286-4ee4-a609-567463f51926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.020507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03628330-9377-4cfc-98d1-532add31e9c8 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.022503 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcdf317-1286-4ee4-a609-567463f51926 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.056520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1007/1998] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:31 2022] POST /identity/v3/projects => generated 361 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:32.057043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03628330-9377-4cfc-98d1-532add31e9c8 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.059977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 992/1999] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:32 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:32.060909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcdf317-1286-4ee4-a609-567463f51926 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.123256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03628330-9377-4cfc-98d1-532add31e9c8 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.128569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b3adc-e139-42fd-81c9-d4a7ba2df69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.129242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b3adc-e139-42fd-81c9-d4a7ba2df69a admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.129524 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b3adc-e139-42fd-81c9-d4a7ba2df69a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.129798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b3adc-e139-42fd-81c9-d4a7ba2df69a admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.130523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b3adc-e139-42fd-81c9-d4a7ba2df69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.132675 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b3adc-e139-42fd-81c9-d4a7ba2df69a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.134517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcdf317-1286-4ee4-a609-567463f51926 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.140652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b494102-51ce-42d5-bc8c-c28dbc70c092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.141528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b494102-51ce-42d5-bc8c-c28dbc70c092 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.142071 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b494102-51ce-42d5-bc8c-c28dbc70c092 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.142353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b494102-51ce-42d5-bc8c-c28dbc70c092 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.143048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1008/2000] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:32.143161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b494102-51ce-42d5-bc8c-c28dbc70c092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.143947 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b3adc-e139-42fd-81c9-d4a7ba2df69a admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.145112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b494102-51ce-42d5-bc8c-c28dbc70c092 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.156729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b3adc-e139-42fd-81c9-d4a7ba2df69a admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.161990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e3b8c-1db6-4501-a0bb-ec636d270385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.163427 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e3b8c-1db6-4501-a0bb-ec636d270385 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.163880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e3b8c-1db6-4501-a0bb-ec636d270385 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.164196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e3b8c-1db6-4501-a0bb-ec636d270385 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.165057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e3b8c-1db6-4501-a0bb-ec636d270385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.167033 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e3b8c-1db6-4501-a0bb-ec636d270385 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.169737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 993/2001] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:32 2022] POST /identity/v3/users => generated 281 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:32.170465 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b494102-51ce-42d5-bc8c-c28dbc70c092 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.183070 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b494102-51ce-42d5-bc8c-c28dbc70c092 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.187988 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1009/2002] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:32 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:32.188643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c1d6b-91d6-4e9e-9d6a-83b6944e133a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.189114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e3b8c-1db6-4501-a0bb-ec636d270385 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.189671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c1d6b-91d6-4e9e-9d6a-83b6944e133a admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.189962 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c1d6b-91d6-4e9e-9d6a-83b6944e133a admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.190238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c1d6b-91d6-4e9e-9d6a-83b6944e133a admin admin] PATH_INFO: `/v3/projects/6de12edcdf3b4404a4bf3adc0633be4c/users/faa4af13525f4d5fa48120c8b6c5f2a9/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.199945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e3b8c-1db6-4501-a0bb-ec636d270385 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.204855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbd174-b4ee-42ac-b958-f94be22177b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.205632 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbd174-b4ee-42ac-b958-f94be22177b7 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.206097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbd174-b4ee-42ac-b958-f94be22177b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.206537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbd174-b4ee-42ac-b958-f94be22177b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.207231 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbd174-b4ee-42ac-b958-f94be22177b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.207678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c1d6b-91d6-4e9e-9d6a-83b6944e133a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6de12edcdf3b4404a4bf3adc0633be4c, user_id=faa4af13525f4d5fa48120c8b6c5f2a9, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.208927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbd174-b4ee-42ac-b958-f94be22177b7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.211219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c1d6b-91d6-4e9e-9d6a-83b6944e133a admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.224181 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1010/2003] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:32.224181 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbd174-b4ee-42ac-b958-f94be22177b7 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.230877 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 994/2004] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:32 2022] PUT /identity/v3/projects/6de12edcdf3b4404a4bf3adc0633be4c/users/faa4af13525f4d5fa48120c8b6c5f2a9/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:32.231706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c1d6b-91d6-4e9e-9d6a-83b6944e133a admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.233691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbd174-b4ee-42ac-b958-f94be22177b7 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.292199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc0c5a-db89-455b-a377-acef48dd3619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.292890 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc0c5a-db89-455b-a377-acef48dd3619 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.293159 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc0c5a-db89-455b-a377-acef48dd3619 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.293436 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc0c5a-db89-455b-a377-acef48dd3619 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.294161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc0c5a-db89-455b-a377-acef48dd3619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.295423 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c1d6b-91d6-4e9e-9d6a-83b6944e133a admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.295756 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc0c5a-db89-455b-a377-acef48dd3619 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.300157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390d087-2058-4ba3-a861-2f40d5491895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.300823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390d087-2058-4ba3-a861-2f40d5491895 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.301101 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390d087-2058-4ba3-a861-2f40d5491895 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.301373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390d087-2058-4ba3-a861-2f40d5491895 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.302089 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390d087-2058-4ba3-a861-2f40d5491895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.304344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390d087-2058-4ba3-a861-2f40d5491895 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.306712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1011/2005] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:32.307914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3c8ef-b998-4828-a143-d0c0a69c8975 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.308228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3c8ef-b998-4828-a143-d0c0a69c8975 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.308511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3c8ef-b998-4828-a143-d0c0a69c8975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.314673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 995/2006] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:32.315527 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390d087-2058-4ba3-a861-2f40d5491895 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.324770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390d087-2058-4ba3-a861-2f40d5491895 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.329234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f1c06-b034-4f3c-8c5c-f6ad88fcd40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.329868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f1c06-b034-4f3c-8c5c-f6ad88fcd40d admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.330143 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f1c06-b034-4f3c-8c5c-f6ad88fcd40d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.330415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f1c06-b034-4f3c-8c5c-f6ad88fcd40d admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.331129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f1c06-b034-4f3c-8c5c-f6ad88fcd40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.332713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f1c06-b034-4f3c-8c5c-f6ad88fcd40d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.334968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-bad3c8ef-b998-4828-a143-d0c0a69c8975 None None] MFA Rules not processed for user `faa4af13525f4d5fa48120c8b6c5f2a9`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:32.336570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-bad3c8ef-b998-4828-a143-d0c0a69c8975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:32.345540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 996/2007] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:32.345540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f1c06-b034-4f3c-8c5c-f6ad88fcd40d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.355045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f1c06-b034-4f3c-8c5c-f6ad88fcd40d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.359701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ad00a-10f6-480f-ba92-0edf15254290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.360628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ad00a-10f6-480f-ba92-0edf15254290 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.360913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ad00a-10f6-480f-ba92-0edf15254290 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.361184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ad00a-10f6-480f-ba92-0edf15254290 admin admin] PATH_INFO: `/v3/projects/14a109e3daa644fdb666401ce46fb328/users/ff4e89f24f204a7ea5997ea4361c545f/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.376235 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ad00a-10f6-480f-ba92-0edf15254290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a109e3daa644fdb666401ce46fb328, user_id=ff4e89f24f204a7ea5997ea4361c545f, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.378855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ad00a-10f6-480f-ba92-0edf15254290 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.393197 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 997/2008] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:32 2022] PUT /identity/v3/projects/14a109e3daa644fdb666401ce46fb328/users/ff4e89f24f204a7ea5997ea4361c545f/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:32.393976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ad00a-10f6-480f-ba92-0edf15254290 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.405785 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1012/2009] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:32.407454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938001e3-349e-4fc2-91fe-910ad98ee726 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.407796 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938001e3-349e-4fc2-91fe-910ad98ee726 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.408105 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938001e3-349e-4fc2-91fe-910ad98ee726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.438127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-938001e3-349e-4fc2-91fe-910ad98ee726 None None] MFA Rules not processed for user `ff4e89f24f204a7ea5997ea4361c545f`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:32.439866 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-938001e3-349e-4fc2-91fe-910ad98ee726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:32.464820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ad00a-10f6-480f-ba92-0edf15254290 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.469440 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abefc5-6ab4-46d4-9263-564aaec52175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.470337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abefc5-6ab4-46d4-9263-564aaec52175 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.470620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abefc5-6ab4-46d4-9263-564aaec52175 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.470902 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abefc5-6ab4-46d4-9263-564aaec52175 admin admin] PATH_INFO: `/v3/projects/4b686593abfc4a4f9b540a4a4a17f356/users/b47c81c3bf8542978f6f486318b4a493/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.486386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abefc5-6ab4-46d4-9263-564aaec52175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b686593abfc4a4f9b540a4a4a17f356, user_id=b47c81c3bf8542978f6f486318b4a493, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.489065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abefc5-6ab4-46d4-9263-564aaec52175 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.501422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 998/2010] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:32 2022] PUT /identity/v3/projects/4b686593abfc4a4f9b540a4a4a17f356/users/b47c81c3bf8542978f6f486318b4a493/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:32.502475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abefc5-6ab4-46d4-9263-564aaec52175 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.504104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a4abefc5-6ab4-46d4-9263-564aaec52175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:32.513748 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1013/2011] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:32.515059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754983b4-d827-4f25-a7c5-0a7e3ab23ec4 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.515351 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754983b4-d827-4f25-a7c5-0a7e3ab23ec4 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.515628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754983b4-d827-4f25-a7c5-0a7e3ab23ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.548748 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-754983b4-d827-4f25-a7c5-0a7e3ab23ec4 None None] MFA Rules not processed for user `b47c81c3bf8542978f6f486318b4a493`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:32.550242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-754983b4-d827-4f25-a7c5-0a7e3ab23ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:32.564202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abefc5-6ab4-46d4-9263-564aaec52175 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.567663 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc13591-b2b0-4b9c-9bb9-4a30946a98a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.568333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc13591-b2b0-4b9c-9bb9-4a30946a98a3 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.568640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc13591-b2b0-4b9c-9bb9-4a30946a98a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.568911 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc13591-b2b0-4b9c-9bb9-4a30946a98a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.613359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1014/2012] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:32.614391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754983b4-d827-4f25-a7c5-0a7e3ab23ec4 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.615772 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-754983b4-d827-4f25-a7c5-0a7e3ab23ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:32.623714 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc13591-b2b0-4b9c-9bb9-4a30946a98a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.625162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc13591-b2b0-4b9c-9bb9-4a30946a98a3 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.626825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754983b4-d827-4f25-a7c5-0a7e3ab23ec4 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.630340 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71543d1c-99ab-43d1-b828-23a9e21f3423 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.630987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71543d1c-99ab-43d1-b828-23a9e21f3423 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.631307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71543d1c-99ab-43d1-b828-23a9e21f3423 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.631532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71543d1c-99ab-43d1-b828-23a9e21f3423 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.633891 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 999/2013] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:32.659781 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc13591-b2b0-4b9c-9bb9-4a30946a98a3 service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.668100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc13591-b2b0-4b9c-9bb9-4a30946a98a3 service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.671716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1adf5d-2d4e-4fd6-aee4-bae8295f8541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.672387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1adf5d-2d4e-4fd6-aee4-bae8295f8541 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.672667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1adf5d-2d4e-4fd6-aee4-bae8295f8541 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.672935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1adf5d-2d4e-4fd6-aee4-bae8295f8541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.681541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1adf5d-2d4e-4fd6-aee4-bae8295f8541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.683005 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1adf5d-2d4e-4fd6-aee4-bae8295f8541 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.687469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71543d1c-99ab-43d1-b828-23a9e21f3423 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.688983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71543d1c-99ab-43d1-b828-23a9e21f3423 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.691687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1000/2014] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:27:32.692516 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1adf5d-2d4e-4fd6-aee4-bae8295f8541 service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.698473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1015/2015] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:32.714268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71543d1c-99ab-43d1-b828-23a9e21f3423 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:32.784535 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1adf5d-2d4e-4fd6-aee4-bae8295f8541 service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.786572 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71543d1c-99ab-43d1-b828-23a9e21f3423 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:32.792146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e2cb9-1cce-46a2-8ce2-1f94d5b186d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.793123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e2cb9-1cce-46a2-8ce2-1f94d5b186d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.794125 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e2cb9-1cce-46a2-8ce2-1f94d5b186d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.794125 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e2cb9-1cce-46a2-8ce2-1f94d5b186d5 admin admin] PATH_INFO: `/v3/users/ff4e89f24f204a7ea5997ea4361c545f` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.795208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e2cb9-1cce-46a2-8ce2-1f94d5b186d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff4e89f24f204a7ea5997ea4361c545f)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.799329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e2cb9-1cce-46a2-8ce2-1f94d5b186d5 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.812817 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0017d2-3248-4a0f-9872-16559bdc5096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:32.813728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0017d2-3248-4a0f-9872-16559bdc5096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.817100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0017d2-3248-4a0f-9872-16559bdc5096 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.817100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0017d2-3248-4a0f-9872-16559bdc5096 admin admin] PATH_INFO: `/v3/users/faa4af13525f4d5fa48120c8b6c5f2a9` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.817100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0017d2-3248-4a0f-9872-16559bdc5096 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faa4af13525f4d5fa48120c8b6c5f2a9)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:32.827860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0017d2-3248-4a0f-9872-16559bdc5096 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:32.875195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f47e2cb9-1cce-46a2-8ce2-1f94d5b186d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff4e89f24f204a7ea5997ea4361c545f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:32.875858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f47e2cb9-1cce-46a2-8ce2-1f94d5b186d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff4e89f24f204a7ea5997ea4361c545f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:32.889837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f47e2cb9-1cce-46a2-8ce2-1f94d5b186d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff4e89f24f204a7ea5997ea4361c545f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:32.897356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f47e2cb9-1cce-46a2-8ce2-1f94d5b186d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff4e89f24f204a7ea5997ea4361c545f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:32.898412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f47e2cb9-1cce-46a2-8ce2-1f94d5b186d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff4e89f24f204a7ea5997ea4361c545f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:32.909220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-df0017d2-3248-4a0f-9872-16559bdc5096 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faa4af13525f4d5fa48120c8b6c5f2a9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:32.910141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-df0017d2-3248-4a0f-9872-16559bdc5096 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faa4af13525f4d5fa48120c8b6c5f2a9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:32.910852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-df0017d2-3248-4a0f-9872-16559bdc5096 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faa4af13525f4d5fa48120c8b6c5f2a9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:32.924064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-df0017d2-3248-4a0f-9872-16559bdc5096 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faa4af13525f4d5fa48120c8b6c5f2a9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:32.937170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1016/2016] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:32 2022] DELETE /identity/v3/users/ff4e89f24f204a7ea5997ea4361c545f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:27:32.937946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-df0017d2-3248-4a0f-9872-16559bdc5096 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faa4af13525f4d5fa48120c8b6c5f2a9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:32.941747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5b031-d763-4449-bc40-558a4467f6cb None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.941992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5b031-d763-4449-bc40-558a4467f6cb None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.943345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5b031-d763-4449-bc40-558a4467f6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.962696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1001/2017] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:32 2022] DELETE /identity/v3/users/faa4af13525f4d5fa48120c8b6c5f2a9 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:32.967568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af938a-9566-46f7-9210-cfd914a4afc4 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:32.968071 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af938a-9566-46f7-9210-cfd914a4afc4 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:32.968558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af938a-9566-46f7-9210-cfd914a4afc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:32.978136 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-52f5b031-d763-4449-bc40-558a4467f6cb None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:32.980818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-52f5b031-d763-4449-bc40-558a4467f6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:32.991933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-56af938a-9566-46f7-9210-cfd914a4afc4 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:32.992979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-56af938a-9566-46f7-9210-cfd914a4afc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:33.093423 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1017/2018] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:33.096014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1002/2019] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:33.110785 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5b031-d763-4449-bc40-558a4467f6cb None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.111361 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af938a-9566-46f7-9210-cfd914a4afc4 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.112606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-52f5b031-d763-4449-bc40-558a4467f6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:33.112926 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-56af938a-9566-46f7-9210-cfd914a4afc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:33.179340 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5b031-d763-4449-bc40-558a4467f6cb None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.183412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3460fb3-4bcf-4b6b-adc9-bf4572e7fed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.184136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af938a-9566-46f7-9210-cfd914a4afc4 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.184253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3460fb3-4bcf-4b6b-adc9-bf4572e7fed0 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.184552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3460fb3-4bcf-4b6b-adc9-bf4572e7fed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.184988 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3460fb3-4bcf-4b6b-adc9-bf4572e7fed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.187934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa16be-752b-4a8d-826a-de3fdd874474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.188608 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa16be-752b-4a8d-826a-de3fdd874474 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.188889 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa16be-752b-4a8d-826a-de3fdd874474 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.189162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa16be-752b-4a8d-826a-de3fdd874474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.194958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3460fb3-4bcf-4b6b-adc9-bf4572e7fed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.196452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3460fb3-4bcf-4b6b-adc9-bf4572e7fed0 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.198597 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa16be-752b-4a8d-826a-de3fdd874474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.200703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa16be-752b-4a8d-826a-de3fdd874474 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.206420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1018/2020] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:27:33.210608 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1003/2021] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:27:33.474020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3460fb3-4bcf-4b6b-adc9-bf4572e7fed0 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.478793 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa16be-752b-4a8d-826a-de3fdd874474 service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.478935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b3460fb3-4bcf-4b6b-adc9-bf4572e7fed0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:33.479041 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-fcfa16be-752b-4a8d-826a-de3fdd874474 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:33.492832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa16be-752b-4a8d-826a-de3fdd874474 service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.500283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3460fb3-4bcf-4b6b-adc9-bf4572e7fed0 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.500475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cde15b-9d76-4279-8ab0-5568b08b0874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.501774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cde15b-9d76-4279-8ab0-5568b08b0874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.502131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cde15b-9d76-4279-8ab0-5568b08b0874 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.502538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cde15b-9d76-4279-8ab0-5568b08b0874 admin admin] PATH_INFO: `/v3/projects/14a109e3daa644fdb666401ce46fb328` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.503745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cde15b-9d76-4279-8ab0-5568b08b0874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a109e3daa644fdb666401ce46fb328)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.506082 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cde15b-9d76-4279-8ab0-5568b08b0874 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.506706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb60950-ecb5-434f-9bf7-4d4c413805ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.507591 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb60950-ecb5-434f-9bf7-4d4c413805ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.508090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb60950-ecb5-434f-9bf7-4d4c413805ee admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.508186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb60950-ecb5-434f-9bf7-4d4c413805ee admin admin] PATH_INFO: `/v3/projects/6de12edcdf3b4404a4bf3adc0633be4c` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.509480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb60950-ecb5-434f-9bf7-4d4c413805ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=6de12edcdf3b4404a4bf3adc0633be4c)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.511380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb60950-ecb5-434f-9bf7-4d4c413805ee admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.556570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-18cde15b-9d76-4279-8ab0-5568b08b0874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a109e3daa644fdb666401ce46fb328', 'request_id': 'req-18cde15b-9d76-4279-8ab0-5568b08b0874'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:33.562610 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-18cde15b-9d76-4279-8ab0-5568b08b0874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a109e3daa644fdb666401ce46fb328', 'request_id': 'req-18cde15b-9d76-4279-8ab0-5568b08b0874'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:33.566405 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-0eb60950-ecb5-434f-9bf7-4d4c413805ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6de12edcdf3b4404a4bf3adc0633be4c', 'request_id': 'req-0eb60950-ecb5-434f-9bf7-4d4c413805ee'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:33.567828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-18cde15b-9d76-4279-8ab0-5568b08b0874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a109e3daa644fdb666401ce46fb328', 'request_id': 'req-18cde15b-9d76-4279-8ab0-5568b08b0874'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:33.572583 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-0eb60950-ecb5-434f-9bf7-4d4c413805ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6de12edcdf3b4404a4bf3adc0633be4c', 'request_id': 'req-0eb60950-ecb5-434f-9bf7-4d4c413805ee'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:33.579512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-0eb60950-ecb5-434f-9bf7-4d4c413805ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6de12edcdf3b4404a4bf3adc0633be4c', 'request_id': 'req-0eb60950-ecb5-434f-9bf7-4d4c413805ee'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:33.580716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-18cde15b-9d76-4279-8ab0-5568b08b0874 admin admin] The token cache is being invalidate because project 14a109e3daa644fdb666401ce46fb328 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:33.581003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-18cde15b-9d76-4279-8ab0-5568b08b0874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:33.581614 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-18cde15b-9d76-4279-8ab0-5568b08b0874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:33.583161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1004/2022] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:33 2022] DELETE /identity/v3/projects/14a109e3daa644fdb666401ce46fb328 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:27:33.589042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb1ded-51f2-4c1f-9133-dc05886d2620 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.589335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb1ded-51f2-4c1f-9133-dc05886d2620 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.589664 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb1ded-51f2-4c1f-9133-dc05886d2620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.591857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-0eb60950-ecb5-434f-9bf7-4d4c413805ee admin admin] The token cache is being invalidate because project 6de12edcdf3b4404a4bf3adc0633be4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:33.592172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-0eb60950-ecb5-434f-9bf7-4d4c413805ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:33.592801 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-0eb60950-ecb5-434f-9bf7-4d4c413805ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:33.594308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1019/2023] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:33 2022] DELETE /identity/v3/projects/6de12edcdf3b4404a4bf3adc0633be4c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:33.599078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b74c73-346b-4440-96c6-53847714c9bd None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.599520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b74c73-346b-4440-96c6-53847714c9bd None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.600444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b74c73-346b-4440-96c6-53847714c9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.608025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-73fb1ded-51f2-4c1f-9133-dc05886d2620 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:33.609214 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-73fb1ded-51f2-4c1f-9133-dc05886d2620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:33.618478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c6b74c73-346b-4440-96c6-53847714c9bd None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:33.619730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c6b74c73-346b-4440-96c6-53847714c9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:33.668457 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1005/2024] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:33.675841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bf9b2-3a7f-4c37-aff4-0d6e6ddfbe48 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.676159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bf9b2-3a7f-4c37-aff4-0d6e6ddfbe48 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.676434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bf9b2-3a7f-4c37-aff4-0d6e6ddfbe48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.680417 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1020/2025] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:33.687780 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501698c-5d31-470d-ac94-e46abfd12a18 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.688113 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501698c-5d31-470d-ac94-e46abfd12a18 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.688397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501698c-5d31-470d-ac94-e46abfd12a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.695036 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-837bf9b2-3a7f-4c37-aff4-0d6e6ddfbe48 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:33.696374 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-837bf9b2-3a7f-4c37-aff4-0d6e6ddfbe48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:33.704338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1006/2026] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:33.705498 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2501698c-5d31-470d-ac94-e46abfd12a18 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:33.706710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2501698c-5d31-470d-ac94-e46abfd12a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:33.708508 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bf9b2-3a7f-4c37-aff4-0d6e6ddfbe48 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.715307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1021/2027] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:33.719256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501698c-5d31-470d-ac94-e46abfd12a18 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.720797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bf9b2-3a7f-4c37-aff4-0d6e6ddfbe48 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.725855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215d268-f9f8-4f54-be6f-b61718288392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.726500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215d268-f9f8-4f54-be6f-b61718288392 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.726782 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215d268-f9f8-4f54-be6f-b61718288392 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.727058 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215d268-f9f8-4f54-be6f-b61718288392 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.727783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215d268-f9f8-4f54-be6f-b61718288392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:33.728130 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215d268-f9f8-4f54-be6f-b61718288392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.728287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501698c-5d31-470d-ac94-e46abfd12a18 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.729701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215d268-f9f8-4f54-be6f-b61718288392 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.732876 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3146cf-e68c-45dc-9442-ed5cf5f3d3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.733512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3146cf-e68c-45dc-9442-ed5cf5f3d3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.733797 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3146cf-e68c-45dc-9442-ed5cf5f3d3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.734073 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3146cf-e68c-45dc-9442-ed5cf5f3d3c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.734789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3146cf-e68c-45dc-9442-ed5cf5f3d3c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:33.735093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3146cf-e68c-45dc-9442-ed5cf5f3d3c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.736696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3146cf-e68c-45dc-9442-ed5cf5f3d3c9 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.744925 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1007/2028] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:27:33.745842 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215d268-f9f8-4f54-be6f-b61718288392 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.753308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1022/2029] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:27:33.754636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3146cf-e68c-45dc-9442-ed5cf5f3d3c9 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.755852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215d268-f9f8-4f54-be6f-b61718288392 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.765256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3146cf-e68c-45dc-9442-ed5cf5f3d3c9 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.765397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8567c4d-e50b-411b-950b-17f0788d64b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.766033 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8567c4d-e50b-411b-950b-17f0788d64b8 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.766317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8567c4d-e50b-411b-950b-17f0788d64b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.766592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8567c4d-e50b-411b-950b-17f0788d64b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.767285 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8567c4d-e50b-411b-950b-17f0788d64b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.769197 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8567c4d-e50b-411b-950b-17f0788d64b8 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.769871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d83dd8-1923-44e6-b5cc-d1057294fbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.770569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d83dd8-1923-44e6-b5cc-d1057294fbc5 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.770847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d83dd8-1923-44e6-b5cc-d1057294fbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.771120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d83dd8-1923-44e6-b5cc-d1057294fbc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.771958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d83dd8-1923-44e6-b5cc-d1057294fbc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.773827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d83dd8-1923-44e6-b5cc-d1057294fbc5 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.796513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1008/2030] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:33 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:33.800677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1023/2031] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:33 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:27:33.800800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8567c4d-e50b-411b-950b-17f0788d64b8 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.805290 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d83dd8-1923-44e6-b5cc-d1057294fbc5 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.861503 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8567c4d-e50b-411b-950b-17f0788d64b8 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.866667 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d83dd8-1923-44e6-b5cc-d1057294fbc5 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.866866 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f3abb-84e5-4b43-8d0d-99135cab747f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.867506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f3abb-84e5-4b43-8d0d-99135cab747f admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.867869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f3abb-84e5-4b43-8d0d-99135cab747f admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.868081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f3abb-84e5-4b43-8d0d-99135cab747f admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.868767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f3abb-84e5-4b43-8d0d-99135cab747f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.870663 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f3abb-84e5-4b43-8d0d-99135cab747f admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.871777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8cd87-bb0d-445c-b4a5-b43e7082aca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.872493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8cd87-bb0d-445c-b4a5-b43e7082aca0 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.872777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8cd87-bb0d-445c-b4a5-b43e7082aca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.873053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8cd87-bb0d-445c-b4a5-b43e7082aca0 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.873748 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8cd87-bb0d-445c-b4a5-b43e7082aca0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.875636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8cd87-bb0d-445c-b4a5-b43e7082aca0 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.893667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1009/2032] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:33 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:27:33.894762 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f3abb-84e5-4b43-8d0d-99135cab747f admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.898774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1024/2033] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:33 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:33.899630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8cd87-bb0d-445c-b4a5-b43e7082aca0 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.904859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f3abb-84e5-4b43-8d0d-99135cab747f admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.909144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8cd87-bb0d-445c-b4a5-b43e7082aca0 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.910107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b573a-7ce4-42d8-b491-045522f6179e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.910764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b573a-7ce4-42d8-b491-045522f6179e admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.911052 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b573a-7ce4-42d8-b491-045522f6179e admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.911330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b573a-7ce4-42d8-b491-045522f6179e admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.913264 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b573a-7ce4-42d8-b491-045522f6179e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.913814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45378fd1-20d9-49ce-9387-82c87ba261fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.914457 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378fd1-20d9-49ce-9387-82c87ba261fd admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.914737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378fd1-20d9-49ce-9387-82c87ba261fd admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.914851 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b573a-7ce4-42d8-b491-045522f6179e admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.915006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378fd1-20d9-49ce-9387-82c87ba261fd admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.915725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45378fd1-20d9-49ce-9387-82c87ba261fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.917322 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45378fd1-20d9-49ce-9387-82c87ba261fd admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.924635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1010/2034] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:33.926970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1025/2035] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:33.929343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b573a-7ce4-42d8-b491-045522f6179e admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.931294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45378fd1-20d9-49ce-9387-82c87ba261fd admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.938904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b573a-7ce4-42d8-b491-045522f6179e admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.940692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45378fd1-20d9-49ce-9387-82c87ba261fd admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.943560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa6f27-4717-4b51-a443-99024032786d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.944894 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa6f27-4717-4b51-a443-99024032786d admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.945441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa6f27-4717-4b51-a443-99024032786d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.945723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa6f27-4717-4b51-a443-99024032786d admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.945812 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124adf73-4ce0-4c6d-b9a4-f7d780f306eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.946434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124adf73-4ce0-4c6d-b9a4-f7d780f306eb admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.946548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa6f27-4717-4b51-a443-99024032786d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.946716 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124adf73-4ce0-4c6d-b9a4-f7d780f306eb admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.946997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124adf73-4ce0-4c6d-b9a4-f7d780f306eb admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.947706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124adf73-4ce0-4c6d-b9a4-f7d780f306eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.948027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa6f27-4717-4b51-a443-99024032786d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.949308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124adf73-4ce0-4c6d-b9a4-f7d780f306eb admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:33.960671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1011/2036] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:33.962251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1026/2037] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:33.963778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa6f27-4717-4b51-a443-99024032786d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.965160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124adf73-4ce0-4c6d-b9a4-f7d780f306eb admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:33.974088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa6f27-4717-4b51-a443-99024032786d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.974784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124adf73-4ce0-4c6d-b9a4-f7d780f306eb admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:33.979318 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c300d30-cc65-4b9e-8ad1-cc02da8b82ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.979524 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ba141-f6a9-40c4-b6c8-cf18f33bb4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:33.980241 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c300d30-cc65-4b9e-8ad1-cc02da8b82ce admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.980480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ba141-f6a9-40c4-b6c8-cf18f33bb4d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:33.980581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c300d30-cc65-4b9e-8ad1-cc02da8b82ce admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.980763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ba141-f6a9-40c4-b6c8-cf18f33bb4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:33.980862 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c300d30-cc65-4b9e-8ad1-cc02da8b82ce admin admin] PATH_INFO: `/v3/projects/fc372fd46c7e41f4aa5cdea2c17bb991/users/d093b9daed0945c5b5e344e380c95c9c/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.981038 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ba141-f6a9-40c4-b6c8-cf18f33bb4d5 admin admin] PATH_INFO: `/v3/projects/c8cbe32e8ede404fa3d19537256606e1/users/dc01367aaf7742d8abdd0fdc4de64f6e/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:33.996353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c300d30-cc65-4b9e-8ad1-cc02da8b82ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc372fd46c7e41f4aa5cdea2c17bb991, user_id=d093b9daed0945c5b5e344e380c95c9c, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.998212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ba141-f6a9-40c4-b6c8-cf18f33bb4d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8cbe32e8ede404fa3d19537256606e1, user_id=dc01367aaf7742d8abdd0fdc4de64f6e, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:33.999167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c300d30-cc65-4b9e-8ad1-cc02da8b82ce admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:34.000931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ba141-f6a9-40c4-b6c8-cf18f33bb4d5 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:34.014780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1012/2038] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:33 2022] PUT /identity/v3/projects/fc372fd46c7e41f4aa5cdea2c17bb991/users/d093b9daed0945c5b5e344e380c95c9c/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:34.015567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1027/2039] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:33 2022] PUT /identity/v3/projects/c8cbe32e8ede404fa3d19537256606e1/users/dc01367aaf7742d8abdd0fdc4de64f6e/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:27:34.020444 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a3091-f2bb-4bb7-b5a3-c626480b2ad4 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:34.029163 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a3091-f2bb-4bb7-b5a3-c626480b2ad4 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:34.029163 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a3091-f2bb-4bb7-b5a3-c626480b2ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:34.029581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b15564-1636-4da5-acfe-f397dbfc65c0 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:34.029581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b15564-1636-4da5-acfe-f397dbfc65c0 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:34.029581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b15564-1636-4da5-acfe-f397dbfc65c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:34.042473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-879a3091-f2bb-4bb7-b5a3-c626480b2ad4 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:34.043670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-879a3091-f2bb-4bb7-b5a3-c626480b2ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:34.061907 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-f3b15564-1636-4da5-acfe-f397dbfc65c0 None None] MFA Rules not processed for user `d093b9daed0945c5b5e344e380c95c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:34.063104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f3b15564-1636-4da5-acfe-f397dbfc65c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:34.138715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1013/2040] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:34.148004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1028/2041] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:34.150433 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a3091-f2bb-4bb7-b5a3-c626480b2ad4 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:34.152233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-879a3091-f2bb-4bb7-b5a3-c626480b2ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:34.172649 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b15564-1636-4da5-acfe-f397dbfc65c0 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:34.174106 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f3b15564-1636-4da5-acfe-f397dbfc65c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:34.228858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a3091-f2bb-4bb7-b5a3-c626480b2ad4 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:34.233131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996d526-ee7d-4b3c-9f20-a317da4f1144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:34.233790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996d526-ee7d-4b3c-9f20-a317da4f1144 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:34.234075 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996d526-ee7d-4b3c-9f20-a317da4f1144 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:34.234356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996d526-ee7d-4b3c-9f20-a317da4f1144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:34.244331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996d526-ee7d-4b3c-9f20-a317da4f1144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:34.244950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b15564-1636-4da5-acfe-f397dbfc65c0 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:34.245750 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996d526-ee7d-4b3c-9f20-a317da4f1144 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:34.248951 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67d933-5a2f-4b21-b559-a5ac16f72e71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:34.249698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67d933-5a2f-4b21-b559-a5ac16f72e71 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:34.250005 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67d933-5a2f-4b21-b559-a5ac16f72e71 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:34.250383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67d933-5a2f-4b21-b559-a5ac16f72e71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:34.255758 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1014/2042] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:34.263812 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67d933-5a2f-4b21-b559-a5ac16f72e71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:34.265292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67d933-5a2f-4b21-b559-a5ac16f72e71 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:34.275373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1029/2043] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:27:38.566167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210eee7-eb61-4f41-8ea7-346dc4bd7722 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:38.566671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210eee7-eb61-4f41-8ea7-346dc4bd7722 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:38.566749 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210eee7-eb61-4f41-8ea7-346dc4bd7722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:38.635338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-7210eee7-eb61-4f41-8ea7-346dc4bd7722 None None] MFA Rules not processed for user `dc01367aaf7742d8abdd0fdc4de64f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:38.638450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-7210eee7-eb61-4f41-8ea7-346dc4bd7722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:38.754299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1015/2044] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:38.755754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67d933-5a2f-4b21-b559-a5ac16f72e71 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:38.769408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67d933-5a2f-4b21-b559-a5ac16f72e71 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:38.777095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74766e-438b-418e-83b9-094b95dd44a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:38.777930 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74766e-438b-418e-83b9-094b95dd44a0 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:38.778226 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74766e-438b-418e-83b9-094b95dd44a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:38.778666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74766e-438b-418e-83b9-094b95dd44a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:38.779946 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74766e-438b-418e-83b9-094b95dd44a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:38.782645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74766e-438b-418e-83b9-094b95dd44a0 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:38.821844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1030/2045] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:38 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:38.823182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210eee7-eb61-4f41-8ea7-346dc4bd7722 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:38.842701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210eee7-eb61-4f41-8ea7-346dc4bd7722 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:38.923320 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc9a04-5de5-4094-816c-92f26d91c33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:38.924042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc9a04-5de5-4094-816c-92f26d91c33e admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:38.924332 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc9a04-5de5-4094-816c-92f26d91c33e admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:38.924614 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc9a04-5de5-4094-816c-92f26d91c33e admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:38.925361 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc9a04-5de5-4094-816c-92f26d91c33e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:38.927534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc9a04-5de5-4094-816c-92f26d91c33e admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:38.963922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1016/2046] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:38 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:27:38.967700 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74766e-438b-418e-83b9-094b95dd44a0 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:38.979360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74766e-438b-418e-83b9-094b95dd44a0 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:38.991113 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b6e44-5c39-4221-8a4a-8359f4eb6101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:38.998360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b6e44-5c39-4221-8a4a-8359f4eb6101 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:38.999013 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b6e44-5c39-4221-8a4a-8359f4eb6101 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:38.999499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b6e44-5c39-4221-8a4a-8359f4eb6101 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:39.000552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b6e44-5c39-4221-8a4a-8359f4eb6101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:39.003139 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b6e44-5c39-4221-8a4a-8359f4eb6101 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:39.028332 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1031/2047] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:27:39.036634 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc9a04-5de5-4094-816c-92f26d91c33e admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:39.057697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc9a04-5de5-4094-816c-92f26d91c33e admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:39.066351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af186db3-7ad6-45a1-9cd0-50d993879b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:39.068976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af186db3-7ad6-45a1-9cd0-50d993879b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:39.069590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af186db3-7ad6-45a1-9cd0-50d993879b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:39.070124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af186db3-7ad6-45a1-9cd0-50d993879b34 admin admin] PATH_INFO: `/v3/projects/04dd4b39fd594b19a4fa84188cf31fd4/users/bb4d0dd751bb4703a48a4e5066085e60/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:39.097287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af186db3-7ad6-45a1-9cd0-50d993879b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04dd4b39fd594b19a4fa84188cf31fd4, user_id=bb4d0dd751bb4703a48a4e5066085e60, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:39.101541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af186db3-7ad6-45a1-9cd0-50d993879b34 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:39.123672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1017/2048] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:39 2022] PUT /identity/v3/projects/04dd4b39fd594b19a4fa84188cf31fd4/users/bb4d0dd751bb4703a48a4e5066085e60/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:39.127846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b6e44-5c39-4221-8a4a-8359f4eb6101 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:39.145421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b6e44-5c39-4221-8a4a-8359f4eb6101 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:39.226719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f18589-aca3-4d85-8466-6bad26de8028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:39.228082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f18589-aca3-4d85-8466-6bad26de8028 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:39.228082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f18589-aca3-4d85-8466-6bad26de8028 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:39.228082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f18589-aca3-4d85-8466-6bad26de8028 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:39.228988 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f18589-aca3-4d85-8466-6bad26de8028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:39.230834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f18589-aca3-4d85-8466-6bad26de8028 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:39.242635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1032/2049] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:27:39.246818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af186db3-7ad6-45a1-9cd0-50d993879b34 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:39.258611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af186db3-7ad6-45a1-9cd0-50d993879b34 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:39.263466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69822c-5ef1-424f-91f5-8ace4e1a68f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:39.264406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69822c-5ef1-424f-91f5-8ace4e1a68f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:39.264692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69822c-5ef1-424f-91f5-8ace4e1a68f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:39.264973 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69822c-5ef1-424f-91f5-8ace4e1a68f3 admin admin] PATH_INFO: `/v3/projects/04dd4b39fd594b19a4fa84188cf31fd4/users/bb4d0dd751bb4703a48a4e5066085e60/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:39.266993 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69822c-5ef1-424f-91f5-8ace4e1a68f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04dd4b39fd594b19a4fa84188cf31fd4, user_id=bb4d0dd751bb4703a48a4e5066085e60, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:39.269823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69822c-5ef1-424f-91f5-8ace4e1a68f3 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:39.282472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1018/2050] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:39 2022] PUT /identity/v3/projects/04dd4b39fd594b19a4fa84188cf31fd4/users/bb4d0dd751bb4703a48a4e5066085e60/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:39.682319 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f18589-aca3-4d85-8466-6bad26de8028 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:39.684362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-28f18589-aca3-4d85-8466-6bad26de8028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:39.697637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f18589-aca3-4d85-8466-6bad26de8028 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:39.770257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:39.771315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:39.772405 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:39.772947 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] PATH_INFO: `/v3/users/7cf9bb405a0e4dd597066240310990f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:39.774439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf9bb405a0e4dd597066240310990f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:39.776754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:39.829017 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf9bb405a0e4dd597066240310990f3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:39.829693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf9bb405a0e4dd597066240310990f3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:39.840030 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf9bb405a0e4dd597066240310990f3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:39.847605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf9bb405a0e4dd597066240310990f3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:39.849010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf9bb405a0e4dd597066240310990f3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:39.882699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1033/2051] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:39 2022] DELETE /identity/v3/users/7cf9bb405a0e4dd597066240310990f3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:39.887448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c6cb8-e6c9-4a71-9ca4-bddf37fd3dcb None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:39.887965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c6cb8-e6c9-4a71-9ca4-bddf37fd3dcb None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:39.888319 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c6cb8-e6c9-4a71-9ca4-bddf37fd3dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:39.908620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-9e4c6cb8-e6c9-4a71-9ca4-bddf37fd3dcb None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:39.910096 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9e4c6cb8-e6c9-4a71-9ca4-bddf37fd3dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:40.018123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1019/2052] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:40.037014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:40.040085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:40.124217 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283fe3d-5b8a-4f6a-9e92-6b6f83133c41 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:40.129684 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24eb64-17c5-4cc0-9a7c-6242aaeaecfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:40.129956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24eb64-17c5-4cc0-9a7c-6242aaeaecfd service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:40.130218 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24eb64-17c5-4cc0-9a7c-6242aaeaecfd service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:40.130525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24eb64-17c5-4cc0-9a7c-6242aaeaecfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:40.142016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24eb64-17c5-4cc0-9a7c-6242aaeaecfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:40.143728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24eb64-17c5-4cc0-9a7c-6242aaeaecfd service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:40.159762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1034/2053] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:40.253215 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c6cb8-e6c9-4a71-9ca4-bddf37fd3dcb None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:40.254679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9e4c6cb8-e6c9-4a71-9ca4-bddf37fd3dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:40.267583 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c6cb8-e6c9-4a71-9ca4-bddf37fd3dcb None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:40.273722 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:40.274553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:40.274841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:40.275108 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] PATH_INFO: `/v3/users/d093b9daed0945c5b5e344e380c95c9c` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:40.276458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d093b9daed0945c5b5e344e380c95c9c)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:40.278556 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:40.338900 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd093b9daed0945c5b5e344e380c95c9c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:40.339914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd093b9daed0945c5b5e344e380c95c9c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:40.340594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd093b9daed0945c5b5e344e380c95c9c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:40.350202 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd093b9daed0945c5b5e344e380c95c9c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:40.360276 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd093b9daed0945c5b5e344e380c95c9c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:40.390961 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1020/2054] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:40 2022] DELETE /identity/v3/users/d093b9daed0945c5b5e344e380c95c9c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:27:40.395835 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291be18-d65f-4c20-82e1-12bce57f798c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:40.396258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291be18-d65f-4c20-82e1-12bce57f798c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:40.396617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291be18-d65f-4c20-82e1-12bce57f798c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:40.416636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-1291be18-d65f-4c20-82e1-12bce57f798c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:40.417898 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1291be18-d65f-4c20-82e1-12bce57f798c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:40.488197 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1035/2055] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:40.504983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:40.511639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:40.610383 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a6214-4294-4e06-94e7-d3266f0f7ed4 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:40.614507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a0761-d5d1-4a59-8085-beff6e226544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:40.615996 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a0761-d5d1-4a59-8085-beff6e226544 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:40.616469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a0761-d5d1-4a59-8085-beff6e226544 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:40.616959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a0761-d5d1-4a59-8085-beff6e226544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:40.628486 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a0761-d5d1-4a59-8085-beff6e226544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:40.630054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a0761-d5d1-4a59-8085-beff6e226544 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:40.642636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1021/2056] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:41.109635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291be18-d65f-4c20-82e1-12bce57f798c None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:41.111230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1291be18-d65f-4c20-82e1-12bce57f798c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:41.133252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291be18-d65f-4c20-82e1-12bce57f798c None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:41.153154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f402b3-8b4f-48a0-a568-42ec09647f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:41.154060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f402b3-8b4f-48a0-a568-42ec09647f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:41.154342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f402b3-8b4f-48a0-a568-42ec09647f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:41.154622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f402b3-8b4f-48a0-a568-42ec09647f0d admin admin] PATH_INFO: `/v3/projects/fc372fd46c7e41f4aa5cdea2c17bb991` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:41.155930 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f402b3-8b4f-48a0-a568-42ec09647f0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc372fd46c7e41f4aa5cdea2c17bb991)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:41.157858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f402b3-8b4f-48a0-a568-42ec09647f0d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:41.229804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a0761-d5d1-4a59-8085-beff6e226544 service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:41.231232 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d23a0761-d5d1-4a59-8085-beff6e226544 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:41.250005 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a0761-d5d1-4a59-8085-beff6e226544 service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:41.256629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-51f402b3-8b4f-48a0-a568-42ec09647f0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc372fd46c7e41f4aa5cdea2c17bb991', 'request_id': 'req-51f402b3-8b4f-48a0-a568-42ec09647f0d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.263838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-51f402b3-8b4f-48a0-a568-42ec09647f0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc372fd46c7e41f4aa5cdea2c17bb991', 'request_id': 'req-51f402b3-8b4f-48a0-a568-42ec09647f0d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.277322 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-51f402b3-8b4f-48a0-a568-42ec09647f0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc372fd46c7e41f4aa5cdea2c17bb991', 'request_id': 'req-51f402b3-8b4f-48a0-a568-42ec09647f0d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.332943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-51f402b3-8b4f-48a0-a568-42ec09647f0d admin admin] The token cache is being invalidate because project fc372fd46c7e41f4aa5cdea2c17bb991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:41.333243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-51f402b3-8b4f-48a0-a568-42ec09647f0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.334569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-51f402b3-8b4f-48a0-a568-42ec09647f0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.337229 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1036/2057] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:41 2022] DELETE /identity/v3/projects/fc372fd46c7e41f4aa5cdea2c17bb991 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:41.343699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b33ae8-f48b-45dc-ae9b-45c012e2624b None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:41.344016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b33ae8-f48b-45dc-ae9b-45c012e2624b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:41.344366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b33ae8-f48b-45dc-ae9b-45c012e2624b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:41.344759 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:41.346089 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:41.346606 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:41.347097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] PATH_INFO: `/v3/projects/4d5d3a868f1044248d30b5f8604f631b` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:41.348643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d5d3a868f1044248d30b5f8604f631b)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:41.352038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:41.368826 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-d8b33ae8-f48b-45dc-ae9b-45c012e2624b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:41.370341 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d8b33ae8-f48b-45dc-ae9b-45c012e2624b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:41.385918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1037/2058] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:41.389625 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c76fb-b2cc-4367-9acb-db0805e49c98 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:41.390386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c76fb-b2cc-4367-9acb-db0805e49c98 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:41.390386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c76fb-b2cc-4367-9acb-db0805e49c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:41.411983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-224c76fb-b2cc-4367-9acb-db0805e49c98 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:41.413883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-224c76fb-b2cc-4367-9acb-db0805e49c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:41.424122 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1038/2059] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:41.429796 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c76fb-b2cc-4367-9acb-db0805e49c98 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:41.441885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c76fb-b2cc-4367-9acb-db0805e49c98 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:41.456180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d5d3a868f1044248d30b5f8604f631b', 'request_id': 'req-b42395ba-5e06-4f02-9dbf-9db86ce777f4'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.468242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d5d3a868f1044248d30b5f8604f631b', 'request_id': 'req-b42395ba-5e06-4f02-9dbf-9db86ce777f4'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.479436 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d5d3a868f1044248d30b5f8604f631b', 'request_id': 'req-b42395ba-5e06-4f02-9dbf-9db86ce777f4'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.498323 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] The token cache is being invalidate because project 4d5d3a868f1044248d30b5f8604f631b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:41.498934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.500950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.504238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1022/2060] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:41 2022] DELETE /identity/v3/projects/4d5d3a868f1044248d30b5f8604f631b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:41.510415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:41.513461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:41.541687 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05d9e2-0e10-4676-bfe5-90c3e1ca53a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:41.542943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05d9e2-0e10-4676-bfe5-90c3e1ca53a3 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:41.544878 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05d9e2-0e10-4676-bfe5-90c3e1ca53a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:41.546895 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05d9e2-0e10-4676-bfe5-90c3e1ca53a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:41.548896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05d9e2-0e10-4676-bfe5-90c3e1ca53a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:41.550415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05d9e2-0e10-4676-bfe5-90c3e1ca53a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:41.556575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05d9e2-0e10-4676-bfe5-90c3e1ca53a3 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:41.587476 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1039/2061] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:27:41.592106 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05d9e2-0e10-4676-bfe5-90c3e1ca53a3 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:41.594888 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42395ba-5e06-4f02-9dbf-9db86ce777f4 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:41.595307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-0f05d9e2-0e10-4676-bfe5-90c3e1ca53a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:41.601945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:41.602442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:41.602896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:41.603004 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] PATH_INFO: `/v3/users/d6ec1c4f5f1f4e598908b22381743beb` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:41.607876 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6ec1c4f5f1f4e598908b22381743beb)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:41.609827 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:41.617093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05d9e2-0e10-4676-bfe5-90c3e1ca53a3 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:41.622298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fde319-684b-458e-8a0f-3709bcc1c934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:41.623369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde319-684b-458e-8a0f-3709bcc1c934 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:41.623369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde319-684b-458e-8a0f-3709bcc1c934 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:41.623544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde319-684b-458e-8a0f-3709bcc1c934 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:41.624627 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fde319-684b-458e-8a0f-3709bcc1c934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:41.626122 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fde319-684b-458e-8a0f-3709bcc1c934 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:41.673034 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1040/2062] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:41 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:41.674770 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fde319-684b-458e-8a0f-3709bcc1c934 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:41.706287 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6ec1c4f5f1f4e598908b22381743beb'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.707616 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6ec1c4f5f1f4e598908b22381743beb'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.709865 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6ec1c4f5f1f4e598908b22381743beb'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.730793 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6ec1c4f5f1f4e598908b22381743beb'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.768044 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6ec1c4f5f1f4e598908b22381743beb'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:41.823373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fde319-684b-458e-8a0f-3709bcc1c934 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:41.830044 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1023/2063] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:41 2022] DELETE /identity/v3/users/d6ec1c4f5f1f4e598908b22381743beb => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:41.830044 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:41.830044 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:41.921210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9438074-287b-4f13-96fa-f299841ca2c4 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:41.939545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f4244-ebcc-4de3-b9ce-b8248c14e144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:41.940488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f4244-ebcc-4de3-b9ce-b8248c14e144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:41.940773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f4244-ebcc-4de3-b9ce-b8248c14e144 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:41.942353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f4244-ebcc-4de3-b9ce-b8248c14e144 admin admin] PATH_INFO: `/v3/users/b47c81c3bf8542978f6f486318b4a493` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:41.947527 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f4244-ebcc-4de3-b9ce-b8248c14e144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b47c81c3bf8542978f6f486318b4a493)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:41.951732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f4244-ebcc-4de3-b9ce-b8248c14e144 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:42.016179 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefba9f-776c-4327-a408-1feddf157d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:42.017118 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefba9f-776c-4327-a408-1feddf157d1e admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:42.017408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefba9f-776c-4327-a408-1feddf157d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:42.017693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefba9f-776c-4327-a408-1feddf157d1e admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:42.019275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefba9f-776c-4327-a408-1feddf157d1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:42.047679 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefba9f-776c-4327-a408-1feddf157d1e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:42.097619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-066f4244-ebcc-4de3-b9ce-b8248c14e144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b47c81c3bf8542978f6f486318b4a493'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:42.098390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-066f4244-ebcc-4de3-b9ce-b8248c14e144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b47c81c3bf8542978f6f486318b4a493'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:42.099353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-066f4244-ebcc-4de3-b9ce-b8248c14e144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b47c81c3bf8542978f6f486318b4a493'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:42.117282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1041/2064] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:41 2022] POST /identity/v3/users => generated 290 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:27:42.117853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-066f4244-ebcc-4de3-b9ce-b8248c14e144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b47c81c3bf8542978f6f486318b4a493'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:42.118647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefba9f-776c-4327-a408-1feddf157d1e admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:42.121523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-aaefba9f-776c-4327-a408-1feddf157d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:42.139344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-066f4244-ebcc-4de3-b9ce-b8248c14e144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b47c81c3bf8542978f6f486318b4a493'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:42.146168 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefba9f-776c-4327-a408-1feddf157d1e admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:42.165564 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f94d5f-b6e6-4267-a519-30debb319120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:42.166266 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f94d5f-b6e6-4267-a519-30debb319120 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:42.166552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f94d5f-b6e6-4267-a519-30debb319120 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:42.166828 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f94d5f-b6e6-4267-a519-30debb319120 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:42.167570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f94d5f-b6e6-4267-a519-30debb319120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:42.169240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f94d5f-b6e6-4267-a519-30debb319120 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:42.187856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1024/2065] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:41 2022] DELETE /identity/v3/users/b47c81c3bf8542978f6f486318b4a493 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:42.196988 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45676e-6f8b-452d-9f7d-8d06d46a81ef None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:42.197304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45676e-6f8b-452d-9f7d-8d06d46a81ef None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:42.197570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45676e-6f8b-452d-9f7d-8d06d46a81ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:42.210226 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1042/2066] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:27:42.211429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f94d5f-b6e6-4267-a519-30debb319120 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:42.238845 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f94d5f-b6e6-4267-a519-30debb319120 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:42.242737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-8e45676e-6f8b-452d-9f7d-8d06d46a81ef None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:42.244088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8e45676e-6f8b-452d-9f7d-8d06d46a81ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:42.416998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764be75-95da-4790-9a9a-1fd710676bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:42.419044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764be75-95da-4790-9a9a-1fd710676bef admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:42.421030 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764be75-95da-4790-9a9a-1fd710676bef admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:42.422083 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764be75-95da-4790-9a9a-1fd710676bef admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:42.423256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764be75-95da-4790-9a9a-1fd710676bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:42.428145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764be75-95da-4790-9a9a-1fd710676bef admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:42.431104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1025/2067] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:42.449915 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45676e-6f8b-452d-9f7d-8d06d46a81ef None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:42.465244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1043/2068] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:42.465859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45676e-6f8b-452d-9f7d-8d06d46a81ef None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:42.471080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0e57e-3898-4b46-a837-e43708397477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:42.472029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0e57e-3898-4b46-a837-e43708397477 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:42.472312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0e57e-3898-4b46-a837-e43708397477 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:42.472592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0e57e-3898-4b46-a837-e43708397477 admin admin] PATH_INFO: `/v3/projects/b6223852af9c4361b99a24e7a49a7734/users/cb34feb512764ec5b9ca25c30c3d7159/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:42.490023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764be75-95da-4790-9a9a-1fd710676bef admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:42.492494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-9764be75-95da-4790-9a9a-1fd710676bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:42.530318 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0e57e-3898-4b46-a837-e43708397477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6223852af9c4361b99a24e7a49a7734, user_id=cb34feb512764ec5b9ca25c30c3d7159, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:42.533086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0e57e-3898-4b46-a837-e43708397477 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:42.581897 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1026/2069] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:42 2022] PUT /identity/v3/projects/b6223852af9c4361b99a24e7a49a7734/users/cb34feb512764ec5b9ca25c30c3d7159/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:42.587069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0e57e-3898-4b46-a837-e43708397477 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:42.593104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-7da0e57e-3898-4b46-a837-e43708397477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:42.699170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764be75-95da-4790-9a9a-1fd710676bef admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:42.797104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0e57e-3898-4b46-a837-e43708397477 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:42.815843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baab921-bdec-4d04-a158-7b6ff0bab4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:42.816910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baab921-bdec-4d04-a158-7b6ff0bab4fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:42.817300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baab921-bdec-4d04-a158-7b6ff0bab4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:42.817683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baab921-bdec-4d04-a158-7b6ff0bab4fa admin admin] PATH_INFO: `/v3/projects/dd49b016a4464610b94e18580f48e004` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:42.819821 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baab921-bdec-4d04-a158-7b6ff0bab4fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd49b016a4464610b94e18580f48e004)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:42.823503 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baab921-bdec-4d04-a158-7b6ff0bab4fa admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:42.836197 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cea9d-64f6-48aa-92bd-2cff47e039e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:42.837030 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cea9d-64f6-48aa-92bd-2cff47e039e4 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:42.837419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cea9d-64f6-48aa-92bd-2cff47e039e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:42.837833 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cea9d-64f6-48aa-92bd-2cff47e039e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:42.877459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cea9d-64f6-48aa-92bd-2cff47e039e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:42.881112 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cea9d-64f6-48aa-92bd-2cff47e039e4 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:42.908805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1044/2070] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:42.910956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b85a39-2980-43f7-850f-88de758d6510 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:42.911353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b85a39-2980-43f7-850f-88de758d6510 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:42.911747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b85a39-2980-43f7-850f-88de758d6510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:42.972925 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3baab921-bdec-4d04-a158-7b6ff0bab4fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd49b016a4464610b94e18580f48e004', 'request_id': 'req-3baab921-bdec-4d04-a158-7b6ff0bab4fa'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:42.976311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-24b85a39-2980-43f7-850f-88de758d6510 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:42.979867 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-24b85a39-2980-43f7-850f-88de758d6510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:42.985510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3baab921-bdec-4d04-a158-7b6ff0bab4fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd49b016a4464610b94e18580f48e004', 'request_id': 'req-3baab921-bdec-4d04-a158-7b6ff0bab4fa'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:43.004046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3baab921-bdec-4d04-a158-7b6ff0bab4fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd49b016a4464610b94e18580f48e004', 'request_id': 'req-3baab921-bdec-4d04-a158-7b6ff0bab4fa'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:43.032404 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3baab921-bdec-4d04-a158-7b6ff0bab4fa admin admin] The token cache is being invalidate because project dd49b016a4464610b94e18580f48e004 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:43.032697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3baab921-bdec-4d04-a158-7b6ff0bab4fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:43.034057 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3baab921-bdec-4d04-a158-7b6ff0bab4fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:43.037406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1027/2071] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:42 2022] DELETE /identity/v3/projects/dd49b016a4464610b94e18580f48e004 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:43.045103 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fda55-edb1-47a4-9462-abbc0567a319 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:43.045353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fda55-edb1-47a4-9462-abbc0567a319 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:43.045837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fda55-edb1-47a4-9462-abbc0567a319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:43.081702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f69fda55-edb1-47a4-9462-abbc0567a319 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:43.100447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f69fda55-edb1-47a4-9462-abbc0567a319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:43.113824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1045/2072] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:43.125267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b85a39-2980-43f7-850f-88de758d6510 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:43.128164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-24b85a39-2980-43f7-850f-88de758d6510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:43.224057 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1028/2073] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:43.249192 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f0e76-4c61-4918-a480-104bac2d303a None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:43.249857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f0e76-4c61-4918-a480-104bac2d303a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:43.249857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f0e76-4c61-4918-a480-104bac2d303a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:43.258362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b85a39-2980-43f7-850f-88de758d6510 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:43.263878 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edfc5c-a28d-4ee7-a15d-99e278232797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:43.264892 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edfc5c-a28d-4ee7-a15d-99e278232797 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:43.265304 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edfc5c-a28d-4ee7-a15d-99e278232797 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:43.265799 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edfc5c-a28d-4ee7-a15d-99e278232797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:43.285782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82edfc5c-a28d-4ee7-a15d-99e278232797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:43.288727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82edfc5c-a28d-4ee7-a15d-99e278232797 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:43.307272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-480f0e76-4c61-4918-a480-104bac2d303a None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:43.308551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-480f0e76-4c61-4918-a480-104bac2d303a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:43.309295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1046/2074] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:27:43.329002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1029/2075] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:43.335758 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edfc5c-a28d-4ee7-a15d-99e278232797 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:43.357706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edfc5c-a28d-4ee7-a15d-99e278232797 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:43.363676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c32965-ea04-4ced-bfb0-ba9ecff12a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:43.364565 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c32965-ea04-4ced-bfb0-ba9ecff12a62 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:43.365468 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c32965-ea04-4ced-bfb0-ba9ecff12a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:43.366802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c32965-ea04-4ced-bfb0-ba9ecff12a62 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:43.367744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c32965-ea04-4ced-bfb0-ba9ecff12a62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:43.378249 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c32965-ea04-4ced-bfb0-ba9ecff12a62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:43.383515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c32965-ea04-4ced-bfb0-ba9ecff12a62 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:43.424738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1047/2076] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:27:43.436158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f0e76-4c61-4918-a480-104bac2d303a None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:43.457152 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f0e76-4c61-4918-a480-104bac2d303a None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:43.475016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2365d-dd1c-47dd-9432-d0a106a26660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:43.478965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2365d-dd1c-47dd-9432-d0a106a26660 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:43.480710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2365d-dd1c-47dd-9432-d0a106a26660 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:43.481168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e2365d-dd1c-47dd-9432-d0a106a26660 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:43.482152 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2365d-dd1c-47dd-9432-d0a106a26660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:43.484641 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e2365d-dd1c-47dd-9432-d0a106a26660 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:43.551945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c32965-ea04-4ced-bfb0-ba9ecff12a62 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:43.554253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1030/2077] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:43 2022] POST /identity/v3/projects => generated 363 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:43.570683 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c32965-ea04-4ced-bfb0-ba9ecff12a62 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:43.677533 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2365d-dd1c-47dd-9432-d0a106a26660 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:43.680775 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-64e2365d-dd1c-47dd-9432-d0a106a26660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:43.778616 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea53cb5-dd24-46f2-8a74-cd4638e47967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:43.779481 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea53cb5-dd24-46f2-8a74-cd4638e47967 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:43.779612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea53cb5-dd24-46f2-8a74-cd4638e47967 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:43.779838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea53cb5-dd24-46f2-8a74-cd4638e47967 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:43.780562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea53cb5-dd24-46f2-8a74-cd4638e47967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:43.782543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea53cb5-dd24-46f2-8a74-cd4638e47967 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:43.823790 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1048/2078] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:43 2022] POST /identity/v3/users => generated 282 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:43.828050 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea53cb5-dd24-46f2-8a74-cd4638e47967 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:43.851788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea53cb5-dd24-46f2-8a74-cd4638e47967 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:43.864406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c912d-fe4e-42d0-8154-3dd6039d1eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:43.864874 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c912d-fe4e-42d0-8154-3dd6039d1eab admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:43.865177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c912d-fe4e-42d0-8154-3dd6039d1eab admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:43.865464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c912d-fe4e-42d0-8154-3dd6039d1eab admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:43.866948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c912d-fe4e-42d0-8154-3dd6039d1eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:43.870685 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c912d-fe4e-42d0-8154-3dd6039d1eab admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:43.876445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e2365d-dd1c-47dd-9432-d0a106a26660 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:43.884422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551e9a3-3bc1-479f-bc5e-e0211a103a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:43.885804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551e9a3-3bc1-479f-bc5e-e0211a103a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:43.886260 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551e9a3-3bc1-479f-bc5e-e0211a103a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:43.886714 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551e9a3-3bc1-479f-bc5e-e0211a103a59 admin admin] PATH_INFO: `/v3/projects/4b686593abfc4a4f9b540a4a4a17f356` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:43.891929 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551e9a3-3bc1-479f-bc5e-e0211a103a59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b686593abfc4a4f9b540a4a4a17f356)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:43.895369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551e9a3-3bc1-479f-bc5e-e0211a103a59 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:43.906811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1049/2079] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:27:43.909469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c912d-fe4e-42d0-8154-3dd6039d1eab admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:43.931663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c912d-fe4e-42d0-8154-3dd6039d1eab admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:43.942882 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbd3d2-1bec-4649-855d-5ad00014f36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:43.943791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbd3d2-1bec-4649-855d-5ad00014f36b admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:43.944121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbd3d2-1bec-4649-855d-5ad00014f36b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:43.944552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbd3d2-1bec-4649-855d-5ad00014f36b admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:43.946605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbd3d2-1bec-4649-855d-5ad00014f36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:43.949435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbd3d2-1bec-4649-855d-5ad00014f36b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:43.968577 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-9551e9a3-3bc1-479f-bc5e-e0211a103a59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b686593abfc4a4f9b540a4a4a17f356', 'request_id': 'req-9551e9a3-3bc1-479f-bc5e-e0211a103a59'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:43.977772 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1050/2080] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:43.980251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbd3d2-1bec-4649-855d-5ad00014f36b admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:43.981503 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-9551e9a3-3bc1-479f-bc5e-e0211a103a59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b686593abfc4a4f9b540a4a4a17f356', 'request_id': 'req-9551e9a3-3bc1-479f-bc5e-e0211a103a59'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:43.996985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbd3d2-1bec-4649-855d-5ad00014f36b admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:43.997830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-9551e9a3-3bc1-479f-bc5e-e0211a103a59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b686593abfc4a4f9b540a4a4a17f356', 'request_id': 'req-9551e9a3-3bc1-479f-bc5e-e0211a103a59'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:44.041426 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-9551e9a3-3bc1-479f-bc5e-e0211a103a59 admin admin] The token cache is being invalidate because project 4b686593abfc4a4f9b540a4a4a17f356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:44.041918 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-9551e9a3-3bc1-479f-bc5e-e0211a103a59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:44.048868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-9551e9a3-3bc1-479f-bc5e-e0211a103a59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:44.056358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1031/2081] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:43 2022] DELETE /identity/v3/projects/4b686593abfc4a4f9b540a4a4a17f356 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:27:44.058075 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12484830-868e-406f-8894-c05673b1e6e7 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:44.058369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12484830-868e-406f-8894-c05673b1e6e7 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:44.058653 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12484830-868e-406f-8894-c05673b1e6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:44.106821 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-12484830-868e-406f-8894-c05673b1e6e7 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:44.108248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-12484830-868e-406f-8894-c05673b1e6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:44.136821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb0532-916c-4814-8e17-589b6bef5f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:44.137726 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb0532-916c-4814-8e17-589b6bef5f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:44.138005 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb0532-916c-4814-8e17-589b6bef5f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:44.138286 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb0532-916c-4814-8e17-589b6bef5f9d admin admin] PATH_INFO: `/v3/projects/f8b614e7548546baa04d9a2fb758bc19/users/2b6f357689974cf8b6b85592a9ed99fb/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:44.187626 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb0532-916c-4814-8e17-589b6bef5f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b614e7548546baa04d9a2fb758bc19, user_id=2b6f357689974cf8b6b85592a9ed99fb, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:44.195371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb0532-916c-4814-8e17-589b6bef5f9d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:44.223774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1051/2082] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:43 2022] PUT /identity/v3/projects/f8b614e7548546baa04d9a2fb758bc19/users/2b6f357689974cf8b6b85592a9ed99fb/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:27:44.234952 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c1e60-73c2-4e7f-b658-2cabca56fdac None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:44.235336 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c1e60-73c2-4e7f-b658-2cabca56fdac None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:44.235505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c1e60-73c2-4e7f-b658-2cabca56fdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:44.247096 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1032/2083] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:44.255292 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6061c-099e-417e-bbb5-3b3e4070552e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:44.256525 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6061c-099e-417e-bbb5-3b3e4070552e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:44.263067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6061c-099e-417e-bbb5-3b3e4070552e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:44.293655 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-db1c1e60-73c2-4e7f-b658-2cabca56fdac None None] MFA Rules not processed for user `2b6f357689974cf8b6b85592a9ed99fb`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:44.295964 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-db1c1e60-73c2-4e7f-b658-2cabca56fdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:44.306890 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-39c6061c-099e-417e-bbb5-3b3e4070552e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:44.308904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-39c6061c-099e-417e-bbb5-3b3e4070552e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:44.404215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1052/2084] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:44 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:44.408268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c1e60-73c2-4e7f-b658-2cabca56fdac None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:44.409737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-db1c1e60-73c2-4e7f-b658-2cabca56fdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:44.527793 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1033/2085] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:44.529394 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6061c-099e-417e-bbb5-3b3e4070552e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:44.561623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6061c-099e-417e-bbb5-3b3e4070552e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:44.567227 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c1e60-73c2-4e7f-b658-2cabca56fdac None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:44.578657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1fb55-95cc-4985-a873-56497cbad40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:44.579353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1fb55-95cc-4985-a873-56497cbad40f admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:44.579637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1fb55-95cc-4985-a873-56497cbad40f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:44.579929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1fb55-95cc-4985-a873-56497cbad40f admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:44.585096 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1fb55-95cc-4985-a873-56497cbad40f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:44.589067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1fb55-95cc-4985-a873-56497cbad40f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:44.594083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b375b-6675-4ea7-a149-dec892a48dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:44.594790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b375b-6675-4ea7-a149-dec892a48dbc admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:44.595078 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b375b-6675-4ea7-a149-dec892a48dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:44.595359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b375b-6675-4ea7-a149-dec892a48dbc admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:44.596136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b375b-6675-4ea7-a149-dec892a48dbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:44.596451 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b375b-6675-4ea7-a149-dec892a48dbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:44.612964 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b375b-6675-4ea7-a149-dec892a48dbc admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:44.635707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1053/2086] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:44 2022] POST /identity/v3/projects => generated 365 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:44.640303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1fb55-95cc-4985-a873-56497cbad40f admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:44.641498 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1034/2087] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:27:44.644294 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b375b-6675-4ea7-a149-dec892a48dbc admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:44.666875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b375b-6675-4ea7-a149-dec892a48dbc admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:44.732460 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1fb55-95cc-4985-a873-56497cbad40f admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:44.738750 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212a2f3-b703-4c1d-a9da-d39e7ab23345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:44.739696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212a2f3-b703-4c1d-a9da-d39e7ab23345 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:44.740057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212a2f3-b703-4c1d-a9da-d39e7ab23345 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:44.740389 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212a2f3-b703-4c1d-a9da-d39e7ab23345 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:44.745711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212a2f3-b703-4c1d-a9da-d39e7ab23345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:44.745711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212a2f3-b703-4c1d-a9da-d39e7ab23345 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:44.791060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1054/2088] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:44 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:44.794916 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212a2f3-b703-4c1d-a9da-d39e7ab23345 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:44.820444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212a2f3-b703-4c1d-a9da-d39e7ab23345 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:44.822698 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8620aa-48ec-4940-bddd-24d0fd9e7ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:44.823632 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8620aa-48ec-4940-bddd-24d0fd9e7ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:44.824984 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8620aa-48ec-4940-bddd-24d0fd9e7ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:44.825886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8620aa-48ec-4940-bddd-24d0fd9e7ae8 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:44.826158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8620aa-48ec-4940-bddd-24d0fd9e7ae8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:44.832539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8620aa-48ec-4940-bddd-24d0fd9e7ae8 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:44.835282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308e904-fd2e-459a-b3e2-ebf59aed0e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:44.836409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308e904-fd2e-459a-b3e2-ebf59aed0e69 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:44.836698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308e904-fd2e-459a-b3e2-ebf59aed0e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:44.836979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308e904-fd2e-459a-b3e2-ebf59aed0e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:44.837904 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308e904-fd2e-459a-b3e2-ebf59aed0e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:44.845325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308e904-fd2e-459a-b3e2-ebf59aed0e69 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:44.861859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1055/2089] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:44.866609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308e904-fd2e-459a-b3e2-ebf59aed0e69 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:44.879396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308e904-fd2e-459a-b3e2-ebf59aed0e69 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:44.885269 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f7bd9-45c6-4aab-96c6-261dd12c6498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:44.886304 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f7bd9-45c6-4aab-96c6-261dd12c6498 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:44.886840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f7bd9-45c6-4aab-96c6-261dd12c6498 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:44.889114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f7bd9-45c6-4aab-96c6-261dd12c6498 admin admin] PATH_INFO: `/v3/projects/938d124ba7954f10941df11167184309/users/9bfb00fdccc44c97a1b0a86ab7721d58/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:44.897393 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1035/2090] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:44 2022] POST /identity/v3/projects => generated 383 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:44.901864 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8620aa-48ec-4940-bddd-24d0fd9e7ae8 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:44.937362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f7bd9-45c6-4aab-96c6-261dd12c6498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=938d124ba7954f10941df11167184309, user_id=9bfb00fdccc44c97a1b0a86ab7721d58, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:44.944455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f7bd9-45c6-4aab-96c6-261dd12c6498 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:44.984145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1056/2091] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:44 2022] PUT /identity/v3/projects/938d124ba7954f10941df11167184309/users/9bfb00fdccc44c97a1b0a86ab7721d58/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:44.985338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f7bd9-45c6-4aab-96c6-261dd12c6498 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:45.058931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8620aa-48ec-4940-bddd-24d0fd9e7ae8 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:45.126672 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f7bd9-45c6-4aab-96c6-261dd12c6498 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:45.150716 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbf8b1-c883-4216-8e25-de541b110ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:45.151571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbf8b1-c883-4216-8e25-de541b110ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:45.151986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbf8b1-c883-4216-8e25-de541b110ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:45.152390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbf8b1-c883-4216-8e25-de541b110ad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:45.153264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbf8b1-c883-4216-8e25-de541b110ad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:45.158311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbf8b1-c883-4216-8e25-de541b110ad6 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:45.182196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1057/2092] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:45.185782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbf8b1-c883-4216-8e25-de541b110ad6 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:45.192656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063b0ae-ee94-43ce-a0a0-539763374dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:45.193430 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063b0ae-ee94-43ce-a0a0-539763374dbf admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:45.194016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063b0ae-ee94-43ce-a0a0-539763374dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:45.194496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063b0ae-ee94-43ce-a0a0-539763374dbf admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:45.195208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063b0ae-ee94-43ce-a0a0-539763374dbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:45.197199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063b0ae-ee94-43ce-a0a0-539763374dbf admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:45.212170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbf8b1-c883-4216-8e25-de541b110ad6 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:45.224922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6f150-2b0d-40c5-8f97-ccc8ae7b41f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:45.226604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6f150-2b0d-40c5-8f97-ccc8ae7b41f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:45.226892 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6f150-2b0d-40c5-8f97-ccc8ae7b41f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:45.227884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6f150-2b0d-40c5-8f97-ccc8ae7b41f4 admin admin] PATH_INFO: `/v3/projects/938d124ba7954f10941df11167184309/users/9bfb00fdccc44c97a1b0a86ab7721d58/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:45.230137 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6f150-2b0d-40c5-8f97-ccc8ae7b41f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=938d124ba7954f10941df11167184309, user_id=9bfb00fdccc44c97a1b0a86ab7721d58, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:45.235387 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6f150-2b0d-40c5-8f97-ccc8ae7b41f4 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:45.248383 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1036/2093] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:44 2022] POST /identity/v3/users => generated 292 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:45.249670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063b0ae-ee94-43ce-a0a0-539763374dbf admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:45.270069 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1058/2094] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:45 2022] PUT /identity/v3/projects/938d124ba7954f10941df11167184309/users/9bfb00fdccc44c97a1b0a86ab7721d58/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:45.277025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063b0ae-ee94-43ce-a0a0-539763374dbf admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:45.285607 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672e329-1397-4265-81e1-28b8a01bb782 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:45.286046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672e329-1397-4265-81e1-28b8a01bb782 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:45.287198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672e329-1397-4265-81e1-28b8a01bb782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:45.321090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e672e329-1397-4265-81e1-28b8a01bb782 None None] MFA Rules not processed for user `9bfb00fdccc44c97a1b0a86ab7721d58`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:45.322397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e672e329-1397-4265-81e1-28b8a01bb782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:45.429254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb22ce6-1686-47c5-b613-eb45bf0c0fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:45.429981 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb22ce6-1686-47c5-b613-eb45bf0c0fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:45.432420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb22ce6-1686-47c5-b613-eb45bf0c0fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:45.432733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb22ce6-1686-47c5-b613-eb45bf0c0fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:45.433484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb22ce6-1686-47c5-b613-eb45bf0c0fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:45.437483 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb22ce6-1686-47c5-b613-eb45bf0c0fc7 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:45.455068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1059/2095] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:45.455989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1037/2096] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:45.463523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672e329-1397-4265-81e1-28b8a01bb782 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:45.474762 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb22ce6-1686-47c5-b613-eb45bf0c0fc7 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:45.480016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5eb22ce6-1686-47c5-b613-eb45bf0c0fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:45.488367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672e329-1397-4265-81e1-28b8a01bb782 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:45.510204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d5075-ea91-4cdd-b986-c7b4eb2b3b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:45.513219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d5075-ea91-4cdd-b986-c7b4eb2b3b03 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:45.513609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d5075-ea91-4cdd-b986-c7b4eb2b3b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:45.513996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d5075-ea91-4cdd-b986-c7b4eb2b3b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:45.514853 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d5075-ea91-4cdd-b986-c7b4eb2b3b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:45.517584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d5075-ea91-4cdd-b986-c7b4eb2b3b03 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:45.554637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1060/2097] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:45.560925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d5075-ea91-4cdd-b986-c7b4eb2b3b03 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:45.583206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d5075-ea91-4cdd-b986-c7b4eb2b3b03 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:45.598917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf7390-bdac-445a-9bb6-31edd7308f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:45.601255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf7390-bdac-445a-9bb6-31edd7308f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:45.601693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf7390-bdac-445a-9bb6-31edd7308f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:45.602116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf7390-bdac-445a-9bb6-31edd7308f39 admin admin] PATH_INFO: `/v3/projects/cf2a64ab3bc1448b8a934e2d103bb161/users/25e4c84bd2174040a20fa9c60ddb6b55/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:45.643775 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf7390-bdac-445a-9bb6-31edd7308f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf2a64ab3bc1448b8a934e2d103bb161, user_id=25e4c84bd2174040a20fa9c60ddb6b55, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:45.647238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf7390-bdac-445a-9bb6-31edd7308f39 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:45.660258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb22ce6-1686-47c5-b613-eb45bf0c0fc7 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:45.671091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe7099-3eca-4f68-a659-44e45c290e3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:45.671919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe7099-3eca-4f68-a659-44e45c290e3e service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:45.675056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe7099-3eca-4f68-a659-44e45c290e3e service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:45.675501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe7099-3eca-4f68-a659-44e45c290e3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:45.678407 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1061/2098] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:45 2022] PUT /identity/v3/projects/cf2a64ab3bc1448b8a934e2d103bb161/users/25e4c84bd2174040a20fa9c60ddb6b55/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:45.683144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8be909-2cc8-446b-a1f2-1b7cf0afa6dd None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:45.683618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8be909-2cc8-446b-a1f2-1b7cf0afa6dd None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:45.684115 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8be909-2cc8-446b-a1f2-1b7cf0afa6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:45.704622 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe7099-3eca-4f68-a659-44e45c290e3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:45.708189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe7099-3eca-4f68-a659-44e45c290e3e service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:45.725259 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-fb8be909-2cc8-446b-a1f2-1b7cf0afa6dd None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:45.731077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-fb8be909-2cc8-446b-a1f2-1b7cf0afa6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:45.762969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1038/2099] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:45.884779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe7099-3eca-4f68-a659-44e45c290e3e service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:45.894852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe7099-3eca-4f68-a659-44e45c290e3e service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:45.919293 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1062/2100] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:45.979081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8be909-2cc8-446b-a1f2-1b7cf0afa6dd None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:45.980545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-fb8be909-2cc8-446b-a1f2-1b7cf0afa6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:46.024611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef003b0-b4df-4aaf-8b8a-34ffbbcd72e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:46.025286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef003b0-b4df-4aaf-8b8a-34ffbbcd72e5 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:46.025570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef003b0-b4df-4aaf-8b8a-34ffbbcd72e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:46.025845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef003b0-b4df-4aaf-8b8a-34ffbbcd72e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:46.137186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8be909-2cc8-446b-a1f2-1b7cf0afa6dd None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:46.140406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef003b0-b4df-4aaf-8b8a-34ffbbcd72e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:46.142009 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef003b0-b4df-4aaf-8b8a-34ffbbcd72e5 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:46.151772 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25caba0-7b41-46b8-92d8-c67160bb9847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:46.153531 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25caba0-7b41-46b8-92d8-c67160bb9847 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:46.153924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25caba0-7b41-46b8-92d8-c67160bb9847 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:46.155109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25caba0-7b41-46b8-92d8-c67160bb9847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:46.166250 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1039/2101] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:45 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:46.189083 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25caba0-7b41-46b8-92d8-c67160bb9847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:46.195242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25caba0-7b41-46b8-92d8-c67160bb9847 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:46.221103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1063/2102] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:27:47.758198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef003b0-b4df-4aaf-8b8a-34ffbbcd72e5 service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:47.760333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0ef003b0-b4df-4aaf-8b8a-34ffbbcd72e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:47.791812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef003b0-b4df-4aaf-8b8a-34ffbbcd72e5 service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:47.805310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27473df5-8618-4024-94e4-d9860c0f9c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:47.806324 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27473df5-8618-4024-94e4-d9860c0f9c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:47.806637 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27473df5-8618-4024-94e4-d9860c0f9c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:47.806896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27473df5-8618-4024-94e4-d9860c0f9c84 admin admin] PATH_INFO: `/v3/users/9461afbc4a9545f087184cb5e8465512` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:47.811877 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27473df5-8618-4024-94e4-d9860c0f9c84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9461afbc4a9545f087184cb5e8465512)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:47.814481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27473df5-8618-4024-94e4-d9860c0f9c84 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:47.981478 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-27473df5-8618-4024-94e4-d9860c0f9c84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9461afbc4a9545f087184cb5e8465512'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:47.982456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-27473df5-8618-4024-94e4-d9860c0f9c84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9461afbc4a9545f087184cb5e8465512'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:47.983928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-27473df5-8618-4024-94e4-d9860c0f9c84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9461afbc4a9545f087184cb5e8465512'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:48.012994 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-27473df5-8618-4024-94e4-d9860c0f9c84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9461afbc4a9545f087184cb5e8465512'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:48.028579 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-27473df5-8618-4024-94e4-d9860c0f9c84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9461afbc4a9545f087184cb5e8465512'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:48.064525 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1040/2103] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:47 2022] DELETE /identity/v3/users/9461afbc4a9545f087184cb5e8465512 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:49.080877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25caba0-7b41-46b8-92d8-c67160bb9847 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:49.134236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25caba0-7b41-46b8-92d8-c67160bb9847 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:49.201749 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dd449-0c89-4147-9937-1ad13689f8e4 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:49.202248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dd449-0c89-4147-9937-1ad13689f8e4 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:49.202368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dd449-0c89-4147-9937-1ad13689f8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:49.282888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd767293-235c-45b3-9e5c-e1c5cd7e9587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:49.284790 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd767293-235c-45b3-9e5c-e1c5cd7e9587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:49.288255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd767293-235c-45b3-9e5c-e1c5cd7e9587 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:49.293087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd767293-235c-45b3-9e5c-e1c5cd7e9587 admin admin] PATH_INFO: `/v3/projects/935900d28d414052836bae2238354dcc` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:49.293291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-cc4dd449-0c89-4147-9937-1ad13689f8e4 None None] MFA Rules not processed for user `bb4d0dd751bb4703a48a4e5066085e60`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:49.294736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd767293-235c-45b3-9e5c-e1c5cd7e9587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=935900d28d414052836bae2238354dcc)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:49.296815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd767293-235c-45b3-9e5c-e1c5cd7e9587 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:49.298905 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-cc4dd449-0c89-4147-9937-1ad13689f8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:49.449374 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fd767293-235c-45b3-9e5c-e1c5cd7e9587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '935900d28d414052836bae2238354dcc', 'request_id': 'req-fd767293-235c-45b3-9e5c-e1c5cd7e9587'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:49.457373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fd767293-235c-45b3-9e5c-e1c5cd7e9587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '935900d28d414052836bae2238354dcc', 'request_id': 'req-fd767293-235c-45b3-9e5c-e1c5cd7e9587'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:49.473985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fd767293-235c-45b3-9e5c-e1c5cd7e9587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '935900d28d414052836bae2238354dcc', 'request_id': 'req-fd767293-235c-45b3-9e5c-e1c5cd7e9587'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:49.492527 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fd767293-235c-45b3-9e5c-e1c5cd7e9587 admin admin] The token cache is being invalidate because project 935900d28d414052836bae2238354dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:27:49.492971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fd767293-235c-45b3-9e5c-e1c5cd7e9587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:49.493682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fd767293-235c-45b3-9e5c-e1c5cd7e9587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:49.495648 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1064/2104] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:27:49 2022] DELETE /identity/v3/projects/935900d28d414052836bae2238354dcc => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:49.500162 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90225e-ca53-46f6-bf52-ebcc1ce2e307 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:49.500830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90225e-ca53-46f6-bf52-ebcc1ce2e307 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:49.501816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90225e-ca53-46f6-bf52-ebcc1ce2e307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:49.528748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1041/2105] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:49 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:49.541507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4dd449-0c89-4147-9937-1ad13689f8e4 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:49.543330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-cc4dd449-0c89-4147-9937-1ad13689f8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:49.546813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-cf90225e-ca53-46f6-bf52-ebcc1ce2e307 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:49.549915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cf90225e-ca53-46f6-bf52-ebcc1ce2e307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:49.679188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1065/2106] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:27:49.682418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e240f3f-3068-42c2-a760-812000cf17a4 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:49.682810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e240f3f-3068-42c2-a760-812000cf17a4 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:49.683183 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e240f3f-3068-42c2-a760-812000cf17a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:49.732331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4dd449-0c89-4147-9937-1ad13689f8e4 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:49.742006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-8e240f3f-3068-42c2-a760-812000cf17a4 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:49.742952 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8e240f3f-3068-42c2-a760-812000cf17a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:49.779788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1066/2107] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:49.784251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e240f3f-3068-42c2-a760-812000cf17a4 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:49.814512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e240f3f-3068-42c2-a760-812000cf17a4 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:49.834660 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a9b51-5188-4095-abe6-7f05c4f08431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:49.835361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a9b51-5188-4095-abe6-7f05c4f08431 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:49.835759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a9b51-5188-4095-abe6-7f05c4f08431 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:49.836433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a9b51-5188-4095-abe6-7f05c4f08431 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:49.837669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a9b51-5188-4095-abe6-7f05c4f08431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:27:49.837818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a9b51-5188-4095-abe6-7f05c4f08431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:49.839605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a9b51-5188-4095-abe6-7f05c4f08431 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:49.873295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1067/2108] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:27:49.874360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a9b51-5188-4095-abe6-7f05c4f08431 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:49.885155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865d596-7ce1-443c-acd7-e2a5389c6ac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:49.885825 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865d596-7ce1-443c-acd7-e2a5389c6ac7 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:49.886105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865d596-7ce1-443c-acd7-e2a5389c6ac7 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:49.886378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865d596-7ce1-443c-acd7-e2a5389c6ac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:49.887620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a9b51-5188-4095-abe6-7f05c4f08431 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:49.894442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9865d596-7ce1-443c-acd7-e2a5389c6ac7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:49.902203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944f579-104e-490b-a390-76a4095f796f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:49.903044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944f579-104e-490b-a390-76a4095f796f admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:49.903336 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944f579-104e-490b-a390-76a4095f796f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:49.903609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944f579-104e-490b-a390-76a4095f796f admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:49.904451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944f579-104e-490b-a390-76a4095f796f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:49.906462 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944f579-104e-490b-a390-76a4095f796f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:49.972946 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1068/2109] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:49 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:27:49.973629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944f579-104e-490b-a390-76a4095f796f admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:50.072563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865d596-7ce1-443c-acd7-e2a5389c6ac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:50.074417 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865d596-7ce1-443c-acd7-e2a5389c6ac7 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:50.098065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1042/2110] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:50.102501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944f579-104e-490b-a390-76a4095f796f admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:50.120209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277990e-bbc5-4656-8dd5-d9c14230811f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:50.120921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277990e-bbc5-4656-8dd5-d9c14230811f admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:50.121204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277990e-bbc5-4656-8dd5-d9c14230811f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:50.121489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277990e-bbc5-4656-8dd5-d9c14230811f admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:50.122213 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277990e-bbc5-4656-8dd5-d9c14230811f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:50.124134 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277990e-bbc5-4656-8dd5-d9c14230811f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:50.163422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1069/2111] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:49 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:27:50.167691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865d596-7ce1-443c-acd7-e2a5389c6ac7 service nova] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:50.192728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865d596-7ce1-443c-acd7-e2a5389c6ac7 service nova] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:50.203300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3e5fb-9b69-4d03-b4c8-4eb8fdadf998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:50.205703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3e5fb-9b69-4d03-b4c8-4eb8fdadf998 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:50.207745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3e5fb-9b69-4d03-b4c8-4eb8fdadf998 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:50.208335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3e5fb-9b69-4d03-b4c8-4eb8fdadf998 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:50.209962 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3e5fb-9b69-4d03-b4c8-4eb8fdadf998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:50.212603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3e5fb-9b69-4d03-b4c8-4eb8fdadf998 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:50.252907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1043/2112] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:50.258978 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277990e-bbc5-4656-8dd5-d9c14230811f admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:50.273220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277990e-bbc5-4656-8dd5-d9c14230811f admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:50.285100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667258e1-2c77-46ef-b13b-5764fab41693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:50.288880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667258e1-2c77-46ef-b13b-5764fab41693 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:50.289383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667258e1-2c77-46ef-b13b-5764fab41693 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:50.289827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667258e1-2c77-46ef-b13b-5764fab41693 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:50.291014 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667258e1-2c77-46ef-b13b-5764fab41693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:50.295751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667258e1-2c77-46ef-b13b-5764fab41693 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:50.313896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1070/2113] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:50.317761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3e5fb-9b69-4d03-b4c8-4eb8fdadf998 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:50.330517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3e5fb-9b69-4d03-b4c8-4eb8fdadf998 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:50.352885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bc7e0-228a-4f27-81b3-f961530e3f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:50.354186 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bc7e0-228a-4f27-81b3-f961530e3f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:50.356151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bc7e0-228a-4f27-81b3-f961530e3f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:50.356665 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bc7e0-228a-4f27-81b3-f961530e3f22 admin admin] PATH_INFO: `/v3/projects/9c83b0a8b88f452f9e12d077e8ac28c8/users/c56a403eeee84720ba765e2d4367df74/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:50.376094 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bc7e0-228a-4f27-81b3-f961530e3f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c83b0a8b88f452f9e12d077e8ac28c8, user_id=c56a403eeee84720ba765e2d4367df74, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:50.379271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bc7e0-228a-4f27-81b3-f961530e3f22 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:50.422258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1044/2114] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:50 2022] PUT /identity/v3/projects/9c83b0a8b88f452f9e12d077e8ac28c8/users/c56a403eeee84720ba765e2d4367df74/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:50.427135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f7789-6a8e-4f47-ba80-f5c4f8f9eab6 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:50.427488 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f7789-6a8e-4f47-ba80-f5c4f8f9eab6 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:50.428038 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f7789-6a8e-4f47-ba80-f5c4f8f9eab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:50.464889 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-3b9f7789-6a8e-4f47-ba80-f5c4f8f9eab6 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:50.469921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3b9f7789-6a8e-4f47-ba80-f5c4f8f9eab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:50.600971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1071/2115] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:50.648508 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bc7e0-228a-4f27-81b3-f961530e3f22 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:50.651404 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-831bc7e0-228a-4f27-81b3-f961530e3f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:50.769339 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bc7e0-228a-4f27-81b3-f961530e3f22 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:50.789412 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a20b2-b054-48b3-aa92-0116d79ff026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:50.790358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a20b2-b054-48b3-aa92-0116d79ff026 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:50.790828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a20b2-b054-48b3-aa92-0116d79ff026 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:50.791257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a20b2-b054-48b3-aa92-0116d79ff026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:50.816977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a20b2-b054-48b3-aa92-0116d79ff026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:50.819061 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a20b2-b054-48b3-aa92-0116d79ff026 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:50.844671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1045/2116] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:27:52.645157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba31cb0-4128-438a-895c-fdbbf586e3f2 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:52.645676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba31cb0-4128-438a-895c-fdbbf586e3f2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:52.645891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba31cb0-4128-438a-895c-fdbbf586e3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:52.709406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-8ba31cb0-4128-438a-895c-fdbbf586e3f2 None None] MFA Rules not processed for user `cb34feb512764ec5b9ca25c30c3d7159`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:52.710902 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8ba31cb0-4128-438a-895c-fdbbf586e3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:52.822602 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1072/2117] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:52.825482 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a20b2-b054-48b3-aa92-0116d79ff026 service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:52.827789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-cc9a20b2-b054-48b3-aa92-0116d79ff026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:52.852690 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a20b2-b054-48b3-aa92-0116d79ff026 service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:52.863464 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de07d1e-a065-4b24-bd9f-532017b26e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:52.864484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de07d1e-a065-4b24-bd9f-532017b26e25 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:52.867483 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de07d1e-a065-4b24-bd9f-532017b26e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:52.867826 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de07d1e-a065-4b24-bd9f-532017b26e25 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:52.868629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de07d1e-a065-4b24-bd9f-532017b26e25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:52.872923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de07d1e-a065-4b24-bd9f-532017b26e25 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:52.947511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba31cb0-4128-438a-895c-fdbbf586e3f2 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:52.959463 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1046/2118] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:52 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:52.963522 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba31cb0-4128-438a-895c-fdbbf586e3f2 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:53.034186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21bf40-db9f-400a-b7cd-a764a85aac32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:53.034877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21bf40-db9f-400a-b7cd-a764a85aac32 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:53.035161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21bf40-db9f-400a-b7cd-a764a85aac32 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:53.035439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21bf40-db9f-400a-b7cd-a764a85aac32 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:53.036173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21bf40-db9f-400a-b7cd-a764a85aac32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:53.038366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21bf40-db9f-400a-b7cd-a764a85aac32 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:53.090193 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1073/2119] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:52 2022] POST /identity/v3/users => generated 296 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:27:53.094827 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de07d1e-a065-4b24-bd9f-532017b26e25 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:53.111468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de07d1e-a065-4b24-bd9f-532017b26e25 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:53.118278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3d270-b448-4809-ac83-e253d36826c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:53.119234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3d270-b448-4809-ac83-e253d36826c6 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:53.119671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3d270-b448-4809-ac83-e253d36826c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:53.120745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3d270-b448-4809-ac83-e253d36826c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:53.121720 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3d270-b448-4809-ac83-e253d36826c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:53.123638 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3d270-b448-4809-ac83-e253d36826c6 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:53.140492 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21bf40-db9f-400a-b7cd-a764a85aac32 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:53.141251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1047/2120] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:53.151967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21bf40-db9f-400a-b7cd-a764a85aac32 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:53.168765 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed11710-b318-40a1-ab56-e5f1531804fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:53.170306 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed11710-b318-40a1-ab56-e5f1531804fa admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:53.170743 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed11710-b318-40a1-ab56-e5f1531804fa admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:53.171168 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed11710-b318-40a1-ab56-e5f1531804fa admin admin] PATH_INFO: `/v3/projects/c54b5ee463964fa289d41b32b9d47efa/users/26beba5551a94841a0c022a02e71aaa6/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:53.203329 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed11710-b318-40a1-ab56-e5f1531804fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c54b5ee463964fa289d41b32b9d47efa, user_id=26beba5551a94841a0c022a02e71aaa6, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:53.209204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed11710-b318-40a1-ab56-e5f1531804fa admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:53.241922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3d270-b448-4809-ac83-e253d36826c6 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:53.242380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1074/2121] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:53 2022] PUT /identity/v3/projects/c54b5ee463964fa289d41b32b9d47efa/users/26beba5551a94841a0c022a02e71aaa6/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:53.257559 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3d270-b448-4809-ac83-e253d36826c6 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:53.384728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c106b-c95d-4a9a-a083-65fb7d697534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:53.385895 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c106b-c95d-4a9a-a083-65fb7d697534 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:53.386308 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c106b-c95d-4a9a-a083-65fb7d697534 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:53.387045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c106b-c95d-4a9a-a083-65fb7d697534 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:53.388050 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c106b-c95d-4a9a-a083-65fb7d697534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:53.389908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c106b-c95d-4a9a-a083-65fb7d697534 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:53.431968 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1048/2122] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:53.435841 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed11710-b318-40a1-ab56-e5f1531804fa admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:53.450768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed11710-b318-40a1-ab56-e5f1531804fa admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:53.459447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1a1fc-4139-4b57-b89a-e908071bfa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:53.466450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1a1fc-4139-4b57-b89a-e908071bfa95 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:53.466789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1a1fc-4139-4b57-b89a-e908071bfa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:53.467084 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1a1fc-4139-4b57-b89a-e908071bfa95 admin admin] PATH_INFO: `/v3/projects/c54b5ee463964fa289d41b32b9d47efa/users/26beba5551a94841a0c022a02e71aaa6/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:53.473504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb1a1fc-4139-4b57-b89a-e908071bfa95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c54b5ee463964fa289d41b32b9d47efa, user_id=26beba5551a94841a0c022a02e71aaa6, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:53.483338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb1a1fc-4139-4b57-b89a-e908071bfa95 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:53.518166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1075/2123] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:53 2022] PUT /identity/v3/projects/c54b5ee463964fa289d41b32b9d47efa/users/26beba5551a94841a0c022a02e71aaa6/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:55.287622 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb99297-613a-41e9-a6c2-9593802f6ae0 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:55.288091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb99297-613a-41e9-a6c2-9593802f6ae0 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:55.288231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb99297-613a-41e9-a6c2-9593802f6ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:55.337377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-5cb99297-613a-41e9-a6c2-9593802f6ae0 None None] MFA Rules not processed for user `dc01367aaf7742d8abdd0fdc4de64f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:55.340169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5cb99297-613a-41e9-a6c2-9593802f6ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:55.582592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1049/2124] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:56.603680 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32f9a7-bb4d-4b24-af4f-35c3aeca3b04 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:56.604684 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32f9a7-bb4d-4b24-af4f-35c3aeca3b04 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:56.605171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32f9a7-bb4d-4b24-af4f-35c3aeca3b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:56.676143 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2d32f9a7-bb4d-4b24-af4f-35c3aeca3b04 None None] MFA Rules not processed for user `25e4c84bd2174040a20fa9c60ddb6b55`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:56.677954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2d32f9a7-bb4d-4b24-af4f-35c3aeca3b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:56.796845 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1076/2125] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:56.801149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb99297-613a-41e9-a6c2-9593802f6ae0 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:56.803843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5cb99297-613a-41e9-a6c2-9593802f6ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:56.910649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb99297-613a-41e9-a6c2-9593802f6ae0 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:56.918612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771ef3d-588d-4848-84f4-c2a3028a2bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:56.919383 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771ef3d-588d-4848-84f4-c2a3028a2bca admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:56.920069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771ef3d-588d-4848-84f4-c2a3028a2bca admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:56.920584 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771ef3d-588d-4848-84f4-c2a3028a2bca admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:56.921493 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771ef3d-588d-4848-84f4-c2a3028a2bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:56.924267 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771ef3d-588d-4848-84f4-c2a3028a2bca admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:56.962946 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1050/2126] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:27:56 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:27:56.963606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32f9a7-bb4d-4b24-af4f-35c3aeca3b04 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:56.974764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32f9a7-bb4d-4b24-af4f-35c3aeca3b04 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:57.092054 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794686c5-b4d7-43fd-81bb-7790e249e198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:57.101768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794686c5-b4d7-43fd-81bb-7790e249e198 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:57.103035 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794686c5-b4d7-43fd-81bb-7790e249e198 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:57.103482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794686c5-b4d7-43fd-81bb-7790e249e198 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:57.104592 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794686c5-b4d7-43fd-81bb-7790e249e198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:57.113926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794686c5-b4d7-43fd-81bb-7790e249e198 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:57.158724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1077/2127] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:27:56 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:27:57.163007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771ef3d-588d-4848-84f4-c2a3028a2bca admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:57.180138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771ef3d-588d-4848-84f4-c2a3028a2bca admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:57.187061 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbaf81-8798-4b1b-a38e-1a2dabae5828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:57.191105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbaf81-8798-4b1b-a38e-1a2dabae5828 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:57.197046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbaf81-8798-4b1b-a38e-1a2dabae5828 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:57.197719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbaf81-8798-4b1b-a38e-1a2dabae5828 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:57.198811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbaf81-8798-4b1b-a38e-1a2dabae5828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:57.201311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbaf81-8798-4b1b-a38e-1a2dabae5828 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:57.214861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1051/2128] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:57.218810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794686c5-b4d7-43fd-81bb-7790e249e198 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:57.246350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794686c5-b4d7-43fd-81bb-7790e249e198 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:57.257872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6926a7-446f-4e29-b71b-03fd52436cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:57.259067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6926a7-446f-4e29-b71b-03fd52436cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:57.259067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6926a7-446f-4e29-b71b-03fd52436cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:57.259368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6926a7-446f-4e29-b71b-03fd52436cb0 admin admin] PATH_INFO: `/v3/projects/678b52347f054baa98342f098b23a741/users/a56f8a317a7a4942a9de8c6abf2da554/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:57.285020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6926a7-446f-4e29-b71b-03fd52436cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=678b52347f054baa98342f098b23a741, user_id=a56f8a317a7a4942a9de8c6abf2da554, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:57.289274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6926a7-446f-4e29-b71b-03fd52436cb0 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:57.309837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1078/2129] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:57 2022] PUT /identity/v3/projects/678b52347f054baa98342f098b23a741/users/a56f8a317a7a4942a9de8c6abf2da554/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:57.312791 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbaf81-8798-4b1b-a38e-1a2dabae5828 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:57.330564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbaf81-8798-4b1b-a38e-1a2dabae5828 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:57.421358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2dfdd2-1b19-4388-ac08-cc26cfe38404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:57.422394 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dfdd2-1b19-4388-ac08-cc26cfe38404 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:57.422780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dfdd2-1b19-4388-ac08-cc26cfe38404 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:57.423210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dfdd2-1b19-4388-ac08-cc26cfe38404 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:57.424352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2dfdd2-1b19-4388-ac08-cc26cfe38404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:57.426553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2dfdd2-1b19-4388-ac08-cc26cfe38404 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:57.440562 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1052/2130] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:27:57.444504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6926a7-446f-4e29-b71b-03fd52436cb0 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:57.457230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6926a7-446f-4e29-b71b-03fd52436cb0 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:57.468565 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205f355-7f65-4541-9b0b-3cf48e8e39d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:57.469931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205f355-7f65-4541-9b0b-3cf48e8e39d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:57.470619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205f355-7f65-4541-9b0b-3cf48e8e39d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:57.471203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205f355-7f65-4541-9b0b-3cf48e8e39d7 admin admin] PATH_INFO: `/v3/projects/678b52347f054baa98342f098b23a741/users/a56f8a317a7a4942a9de8c6abf2da554/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:57.473986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205f355-7f65-4541-9b0b-3cf48e8e39d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=678b52347f054baa98342f098b23a741, user_id=a56f8a317a7a4942a9de8c6abf2da554, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:57.477983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205f355-7f65-4541-9b0b-3cf48e8e39d7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:57.498880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1079/2131] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:27:57 2022] PUT /identity/v3/projects/678b52347f054baa98342f098b23a741/users/a56f8a317a7a4942a9de8c6abf2da554/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:57.637632 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2dfdd2-1b19-4388-ac08-cc26cfe38404 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:57.650328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8b2dfdd2-1b19-4388-ac08-cc26cfe38404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:57.689938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2dfdd2-1b19-4388-ac08-cc26cfe38404 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:57.833942 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:57.837435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:57.837725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:57.853806 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] PATH_INFO: `/v3/users/2b6f357689974cf8b6b85592a9ed99fb` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:57.857989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b6f357689974cf8b6b85592a9ed99fb)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:57.871876 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:58.015958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b6f357689974cf8b6b85592a9ed99fb'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:58.017259 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b6f357689974cf8b6b85592a9ed99fb'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:58.018712 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b6f357689974cf8b6b85592a9ed99fb'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:58.054546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b6f357689974cf8b6b85592a9ed99fb'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:58.063714 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b6f357689974cf8b6b85592a9ed99fb'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:27:58.087585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1053/2132] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:27:57 2022] DELETE /identity/v3/users/2b6f357689974cf8b6b85592a9ed99fb => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:27:58.092382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570c027-7e95-4b0a-b0e0-35ef69135eb2 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:58.092962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570c027-7e95-4b0a-b0e0-35ef69135eb2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:58.092962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570c027-7e95-4b0a-b0e0-35ef69135eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:58.132518 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2570c027-7e95-4b0a-b0e0-35ef69135eb2 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:27:58.136910 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2570c027-7e95-4b0a-b0e0-35ef69135eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:58.302682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1080/2133] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:27:58.514785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:27:58.517460 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:27:58.612213 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59731b-a51e-4ed9-b551-7d93cdd12739 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:27:58.656484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a3f8-73e5-448d-aa93-7b67bf506257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:27:58.657189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a3f8-73e5-448d-aa93-7b67bf506257 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:27:58.657478 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a3f8-73e5-448d-aa93-7b67bf506257 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:27:58.657758 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a3f8-73e5-448d-aa93-7b67bf506257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:27:58.698915 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154a3f8-73e5-448d-aa93-7b67bf506257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:27:58.699770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154a3f8-73e5-448d-aa93-7b67bf506257 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:27:58.744279 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1054/2134] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:28:00.546873 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570c027-7e95-4b0a-b0e0-35ef69135eb2 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:00.554023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2570c027-7e95-4b0a-b0e0-35ef69135eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:00.575576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570c027-7e95-4b0a-b0e0-35ef69135eb2 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:00.617027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:00.619725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:00.620192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:00.624606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] PATH_INFO: `/v3/projects/f8b614e7548546baa04d9a2fb758bc19` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:00.626221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b614e7548546baa04d9a2fb758bc19)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:00.635140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:00.795825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b614e7548546baa04d9a2fb758bc19', 'request_id': 'req-3cb143d1-76e4-4341-967b-8121c53ce216'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:00.819103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b614e7548546baa04d9a2fb758bc19', 'request_id': 'req-3cb143d1-76e4-4341-967b-8121c53ce216'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:00.844664 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b614e7548546baa04d9a2fb758bc19', 'request_id': 'req-3cb143d1-76e4-4341-967b-8121c53ce216'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:00.867606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] The token cache is being invalidate because project f8b614e7548546baa04d9a2fb758bc19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:28:00.867924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:00.869165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:00.874025 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1081/2135] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:28:00 2022] DELETE /identity/v3/projects/f8b614e7548546baa04d9a2fb758bc19 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:00.877798 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a3f8-73e5-448d-aa93-7b67bf506257 service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:00.915578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a3f8-73e5-448d-aa93-7b67bf506257 service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:01.078212 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c9173-b5b7-40e7-b1cd-209674017ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:01.079388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c9173-b5b7-40e7-b1cd-209674017ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:01.079819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c9173-b5b7-40e7-b1cd-209674017ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:01.080291 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c9173-b5b7-40e7-b1cd-209674017ee8 admin admin] PATH_INFO: `/v3/users/9bfb00fdccc44c97a1b0a86ab7721d58` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:01.081671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c9173-b5b7-40e7-b1cd-209674017ee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bfb00fdccc44c97a1b0a86ab7721d58)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:01.084419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c9173-b5b7-40e7-b1cd-209674017ee8 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:01.239780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-363c9173-b5b7-40e7-b1cd-209674017ee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bfb00fdccc44c97a1b0a86ab7721d58'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:01.241416 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-363c9173-b5b7-40e7-b1cd-209674017ee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bfb00fdccc44c97a1b0a86ab7721d58'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:01.242908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-363c9173-b5b7-40e7-b1cd-209674017ee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bfb00fdccc44c97a1b0a86ab7721d58'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:01.274320 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-363c9173-b5b7-40e7-b1cd-209674017ee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bfb00fdccc44c97a1b0a86ab7721d58'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:01.292245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-363c9173-b5b7-40e7-b1cd-209674017ee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bfb00fdccc44c97a1b0a86ab7721d58'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:01.354753 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1055/2136] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:28:00 2022] DELETE /identity/v3/users/9bfb00fdccc44c97a1b0a86ab7721d58 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:02.689124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:02.691173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:02.746831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb143d1-76e4-4341-967b-8121c53ce216 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:02.856350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7754a3f-a836-4067-963a-21c5710964bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:02.857586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7754a3f-a836-4067-963a-21c5710964bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:02.859960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7754a3f-a836-4067-963a-21c5710964bf admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:02.860474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7754a3f-a836-4067-963a-21c5710964bf admin admin] PATH_INFO: `/v3/projects/938d124ba7954f10941df11167184309` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:02.864254 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7754a3f-a836-4067-963a-21c5710964bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=938d124ba7954f10941df11167184309)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:02.867474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7754a3f-a836-4067-963a-21c5710964bf admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:02.946901 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e7754a3f-a836-4067-963a-21c5710964bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '938d124ba7954f10941df11167184309', 'request_id': 'req-e7754a3f-a836-4067-963a-21c5710964bf'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:02.957293 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e7754a3f-a836-4067-963a-21c5710964bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '938d124ba7954f10941df11167184309', 'request_id': 'req-e7754a3f-a836-4067-963a-21c5710964bf'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:02.972956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e7754a3f-a836-4067-963a-21c5710964bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '938d124ba7954f10941df11167184309', 'request_id': 'req-e7754a3f-a836-4067-963a-21c5710964bf'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:02.987658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e7754a3f-a836-4067-963a-21c5710964bf admin admin] The token cache is being invalidate because project 938d124ba7954f10941df11167184309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:28:02.988078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e7754a3f-a836-4067-963a-21c5710964bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:02.989193 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e7754a3f-a836-4067-963a-21c5710964bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:02.994353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1082/2137] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:28:02 2022] DELETE /identity/v3/projects/938d124ba7954f10941df11167184309 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:02.996060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc51dd-4181-4e28-8146-d036a013a742 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:02.996565 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc51dd-4181-4e28-8146-d036a013a742 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:02.996651 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc51dd-4181-4e28-8146-d036a013a742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:03.019496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-2ebc51dd-4181-4e28-8146-d036a013a742 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:03.020795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-2ebc51dd-4181-4e28-8146-d036a013a742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:03.210439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1056/2138] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:03.219641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985a900-7155-41a0-ae88-c85dce9c9196 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:03.221512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985a900-7155-41a0-ae88-c85dce9c9196 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:03.221989 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985a900-7155-41a0-ae88-c85dce9c9196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:03.253108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-f985a900-7155-41a0-ae88-c85dce9c9196 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:03.257414 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f985a900-7155-41a0-ae88-c85dce9c9196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:03.280293 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1083/2139] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:03.284626 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc51dd-4181-4e28-8146-d036a013a742 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:03.310071 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc51dd-4181-4e28-8146-d036a013a742 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:03.339487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaa3b8-bfa9-4812-9515-af00522bcc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:03.340340 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaa3b8-bfa9-4812-9515-af00522bcc28 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:03.340691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaa3b8-bfa9-4812-9515-af00522bcc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:03.341035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaa3b8-bfa9-4812-9515-af00522bcc28 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:03.342712 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebaa3b8-bfa9-4812-9515-af00522bcc28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:28:03.343098 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebaa3b8-bfa9-4812-9515-af00522bcc28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:03.348578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebaa3b8-bfa9-4812-9515-af00522bcc28 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:03.380520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1057/2140] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:28:03.382847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985a900-7155-41a0-ae88-c85dce9c9196 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:03.402685 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985a900-7155-41a0-ae88-c85dce9c9196 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:03.417833 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee27c7-c18d-415c-ab02-625092dcb5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:03.418914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee27c7-c18d-415c-ab02-625092dcb5bf admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:03.419352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee27c7-c18d-415c-ab02-625092dcb5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:03.423544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee27c7-c18d-415c-ab02-625092dcb5bf admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:03.427238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee27c7-c18d-415c-ab02-625092dcb5bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:03.434016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee27c7-c18d-415c-ab02-625092dcb5bf admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:03.510457 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1084/2141] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:28:03 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:28:03.515075 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaa3b8-bfa9-4812-9515-af00522bcc28 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:03.555700 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaa3b8-bfa9-4812-9515-af00522bcc28 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:03.704658 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbb564-287e-4b4d-98d9-6bdfce6d4dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:03.705844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbb564-287e-4b4d-98d9-6bdfce6d4dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:03.706538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbb564-287e-4b4d-98d9-6bdfce6d4dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:03.707888 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbb564-287e-4b4d-98d9-6bdfce6d4dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:03.711696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbb564-287e-4b4d-98d9-6bdfce6d4dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:03.720079 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbb564-287e-4b4d-98d9-6bdfce6d4dd0 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:03.772506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1058/2142] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:28:03 2022] POST /identity/v3/users => generated 284 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:28:03.776795 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee27c7-c18d-415c-ab02-625092dcb5bf admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:03.798690 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db64cea-9268-46d7-b03b-615da91f99c1 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:03.799933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db64cea-9268-46d7-b03b-615da91f99c1 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:03.800360 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db64cea-9268-46d7-b03b-615da91f99c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:03.801379 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee27c7-c18d-415c-ab02-625092dcb5bf admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:03.814301 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bd8bd-6418-47a6-ab50-f17e5e7116ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:03.815495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bd8bd-6418-47a6-ab50-f17e5e7116ff admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:03.816185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bd8bd-6418-47a6-ab50-f17e5e7116ff admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:03.816624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bd8bd-6418-47a6-ab50-f17e5e7116ff admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:03.817559 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bd8bd-6418-47a6-ab50-f17e5e7116ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:03.827716 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bd8bd-6418-47a6-ab50-f17e5e7116ff admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:03.847604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1085/2143] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:28:03.853311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bd8bd-6418-47a6-ab50-f17e5e7116ff admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:03.865774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bd8bd-6418-47a6-ab50-f17e5e7116ff admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:03.872824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd4bb2-906f-4212-b705-9fe244f84aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:03.874093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd4bb2-906f-4212-b705-9fe244f84aac admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:03.874809 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd4bb2-906f-4212-b705-9fe244f84aac admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:03.875226 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd4bb2-906f-4212-b705-9fe244f84aac admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:03.876342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd4bb2-906f-4212-b705-9fe244f84aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:03.878812 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd4bb2-906f-4212-b705-9fe244f84aac admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:03.891440 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1086/2144] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:28:03.898859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd4bb2-906f-4212-b705-9fe244f84aac admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:03.911405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-4db64cea-9268-46d7-b03b-615da91f99c1 None None] MFA Rules not processed for user `c56a403eeee84720ba765e2d4367df74`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:03.915541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-4db64cea-9268-46d7-b03b-615da91f99c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:03.933478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd4bb2-906f-4212-b705-9fe244f84aac admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:03.942187 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b750f0-e20a-4d19-9ab1-cd2ca169cb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:03.944816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b750f0-e20a-4d19-9ab1-cd2ca169cb8d admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:03.945222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b750f0-e20a-4d19-9ab1-cd2ca169cb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:03.945610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b750f0-e20a-4d19-9ab1-cd2ca169cb8d admin admin] PATH_INFO: `/v3/projects/310edc8624594bbb8d8e55f063e44688/users/b9f0d68834c74a55b77b99edb36a746d/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:03.970447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b750f0-e20a-4d19-9ab1-cd2ca169cb8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=310edc8624594bbb8d8e55f063e44688, user_id=b9f0d68834c74a55b77b99edb36a746d, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:03.970447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b750f0-e20a-4d19-9ab1-cd2ca169cb8d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:04.006151 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1087/2145] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:28:03 2022] PUT /identity/v3/projects/310edc8624594bbb8d8e55f063e44688/users/b9f0d68834c74a55b77b99edb36a746d/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:04.009391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9030-bb1f-4028-96f5-f4374a5c6077 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:04.009391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9030-bb1f-4028-96f5-f4374a5c6077 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:04.009622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9030-bb1f-4028-96f5-f4374a5c6077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:04.089564 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-5bcf9030-bb1f-4028-96f5-f4374a5c6077 None None] MFA Rules not processed for user `b9f0d68834c74a55b77b99edb36a746d`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:04.091156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5bcf9030-bb1f-4028-96f5-f4374a5c6077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:04.197342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1059/2146] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:04.211978 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db64cea-9268-46d7-b03b-615da91f99c1 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:04.214776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-4db64cea-9268-46d7-b03b-615da91f99c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:04.244548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1088/2147] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:04.248681 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf9030-bb1f-4028-96f5-f4374a5c6077 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:04.401108 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db64cea-9268-46d7-b03b-615da91f99c1 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:04.433134 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf9030-bb1f-4028-96f5-f4374a5c6077 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:04.438665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1546f6-e668-41c7-8f13-a8ea3a225bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:04.439414 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1546f6-e668-41c7-8f13-a8ea3a225bcd admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:04.439809 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1546f6-e668-41c7-8f13-a8ea3a225bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:04.440183 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1546f6-e668-41c7-8f13-a8ea3a225bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:04.440969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1546f6-e668-41c7-8f13-a8ea3a225bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:04.443605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1546f6-e668-41c7-8f13-a8ea3a225bcd admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:04.502207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1089/2148] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:28:04 2022] POST /identity/v3/projects => generated 367 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:28:04.509621 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1546f6-e668-41c7-8f13-a8ea3a225bcd admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:04.550231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9366e-904e-48c6-acc6-18b7c6982556 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:04.550829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9366e-904e-48c6-acc6-18b7c6982556 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:04.551115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9366e-904e-48c6-acc6-18b7c6982556 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:04.551396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9366e-904e-48c6-acc6-18b7c6982556 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:04.656268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1546f6-e668-41c7-8f13-a8ea3a225bcd admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:04.668582 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16485e97-0eb6-4417-a297-0f50e3d446f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:04.669287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16485e97-0eb6-4417-a297-0f50e3d446f9 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:04.669566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16485e97-0eb6-4417-a297-0f50e3d446f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:04.669836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16485e97-0eb6-4417-a297-0f50e3d446f9 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:04.670536 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16485e97-0eb6-4417-a297-0f50e3d446f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:04.672516 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16485e97-0eb6-4417-a297-0f50e3d446f9 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:04.688525 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9366e-904e-48c6-acc6-18b7c6982556 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:04.690132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9366e-904e-48c6-acc6-18b7c6982556 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:04.703837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1060/2149] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:28:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:28:04.720920 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9366e-904e-48c6-acc6-18b7c6982556 service nova] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:04.723127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1090/2150] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:28:04 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:28:04.735941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9366e-904e-48c6-acc6-18b7c6982556 service nova] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:04.749728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef85ef-56b8-4351-ae54-61859573a0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:04.752368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef85ef-56b8-4351-ae54-61859573a0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:04.752867 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef85ef-56b8-4351-ae54-61859573a0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:04.753434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef85ef-56b8-4351-ae54-61859573a0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:04.754327 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef85ef-56b8-4351-ae54-61859573a0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:04.756553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef85ef-56b8-4351-ae54-61859573a0d7 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:04.790530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1061/2151] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:28:04.791835 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16485e97-0eb6-4417-a297-0f50e3d446f9 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:04.805554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16485e97-0eb6-4417-a297-0f50e3d446f9 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:04.822464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d299e1-7866-4811-886f-9d219c79966a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:04.824096 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d299e1-7866-4811-886f-9d219c79966a admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:04.824391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d299e1-7866-4811-886f-9d219c79966a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:04.824689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d299e1-7866-4811-886f-9d219c79966a admin admin] PATH_INFO: `/v3/projects/3083351ddf8b4e94a3529ba2af7c5211/users/6f23e45ec82c4862a96d13abba3a5245/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:04.883087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d299e1-7866-4811-886f-9d219c79966a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3083351ddf8b4e94a3529ba2af7c5211, user_id=6f23e45ec82c4862a96d13abba3a5245, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:04.886216 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d299e1-7866-4811-886f-9d219c79966a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:04.913054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef85ef-56b8-4351-ae54-61859573a0d7 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:04.914057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1091/2152] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:28:04 2022] PUT /identity/v3/projects/3083351ddf8b4e94a3529ba2af7c5211/users/6f23e45ec82c4862a96d13abba3a5245/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:04.951427 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef85ef-56b8-4351-ae54-61859573a0d7 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:05.149222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dba46-00a4-4136-ad77-558e6c0ec3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:05.154105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dba46-00a4-4136-ad77-558e6c0ec3ea admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:05.154391 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dba46-00a4-4136-ad77-558e6c0ec3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:05.154703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dba46-00a4-4136-ad77-558e6c0ec3ea admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:05.155462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dba46-00a4-4136-ad77-558e6c0ec3ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:05.159204 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dba46-00a4-4136-ad77-558e6c0ec3ea admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:05.193439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d299e1-7866-4811-886f-9d219c79966a admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:05.194870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1062/2153] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:28:05.237721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d299e1-7866-4811-886f-9d219c79966a admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:05.248269 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e66d1-d588-40a7-a5d3-bafff3950cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:05.251245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e66d1-d588-40a7-a5d3-bafff3950cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:05.251576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e66d1-d588-40a7-a5d3-bafff3950cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:05.251856 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e66d1-d588-40a7-a5d3-bafff3950cd6 admin admin] PATH_INFO: `/v3/projects/3083351ddf8b4e94a3529ba2af7c5211/users/6f23e45ec82c4862a96d13abba3a5245/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:05.272897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e66d1-d588-40a7-a5d3-bafff3950cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3083351ddf8b4e94a3529ba2af7c5211, user_id=6f23e45ec82c4862a96d13abba3a5245, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:05.277252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e66d1-d588-40a7-a5d3-bafff3950cd6 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:05.331880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1092/2154] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:28:05 2022] PUT /identity/v3/projects/3083351ddf8b4e94a3529ba2af7c5211/users/6f23e45ec82c4862a96d13abba3a5245/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:05.347120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0fa57-950a-4742-836c-0d537b4a7233 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:05.347662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0fa57-950a-4742-836c-0d537b4a7233 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:05.351592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0fa57-950a-4742-836c-0d537b4a7233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:05.437607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-47c0fa57-950a-4742-836c-0d537b4a7233 None None] MFA Rules not processed for user `6f23e45ec82c4862a96d13abba3a5245`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:05.438968 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-47c0fa57-950a-4742-836c-0d537b4a7233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:05.693932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e66d1-d588-40a7-a5d3-bafff3950cd6 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:05.697611 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-fa8e66d1-d588-40a7-a5d3-bafff3950cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:05.713851 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1063/2155] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:05.864810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e66d1-d588-40a7-a5d3-bafff3950cd6 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:05.879744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d67ded-0779-43d4-9d1d-43aec69867b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:05.882816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d67ded-0779-43d4-9d1d-43aec69867b6 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:05.883757 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d67ded-0779-43d4-9d1d-43aec69867b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:05.884408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d67ded-0779-43d4-9d1d-43aec69867b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:05.911110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d67ded-0779-43d4-9d1d-43aec69867b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:05.914106 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d67ded-0779-43d4-9d1d-43aec69867b6 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:05.938877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1093/2156] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:28:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:28:06.012718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0fa57-950a-4742-836c-0d537b4a7233 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:06.016067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-47c0fa57-950a-4742-836c-0d537b4a7233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:06.035236 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0fa57-950a-4742-836c-0d537b4a7233 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:06.043264 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99451907-66cb-466a-a642-c089796c9196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:06.044350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99451907-66cb-466a-a642-c089796c9196 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:06.044852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99451907-66cb-466a-a642-c089796c9196 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:06.046372 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99451907-66cb-466a-a642-c089796c9196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:06.190644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99451907-66cb-466a-a642-c089796c9196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:06.196945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99451907-66cb-466a-a642-c089796c9196 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:06.221444 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1064/2157] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:28:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:28:07.783609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf8076-d69f-4241-936f-63d762b930b4 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:07.784848 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf8076-d69f-4241-936f-63d762b930b4 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:07.785362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf8076-d69f-4241-936f-63d762b930b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:07.824343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-44bf8076-d69f-4241-936f-63d762b930b4 None None] MFA Rules not processed for user `c56a403eeee84720ba765e2d4367df74`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:07.826914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-44bf8076-d69f-4241-936f-63d762b930b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:08.059521 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1094/2158] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:08.523513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af2930-474d-4a88-9788-06496dd3383e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:08.523513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af2930-474d-4a88-9788-06496dd3383e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:08.523513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af2930-474d-4a88-9788-06496dd3383e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:08.606973 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-86af2930-474d-4a88-9788-06496dd3383e None None] MFA Rules not processed for user `26beba5551a94841a0c022a02e71aaa6`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:08.606973 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-86af2930-474d-4a88-9788-06496dd3383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:08.899396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1065/2159] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:08.902985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf8076-d69f-4241-936f-63d762b930b4 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:09.135277 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf8076-d69f-4241-936f-63d762b930b4 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:09.141650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91799ac8-448f-4ca8-9733-9f974be9e51e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:09.142640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91799ac8-448f-4ca8-9733-9f974be9e51e service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:09.143079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91799ac8-448f-4ca8-9733-9f974be9e51e service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:09.143510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91799ac8-448f-4ca8-9733-9f974be9e51e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:09.147164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-91799ac8-448f-4ca8-9733-9f974be9e51e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:09.246985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91799ac8-448f-4ca8-9733-9f974be9e51e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:09.249803 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91799ac8-448f-4ca8-9733-9f974be9e51e service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:09.266349 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1095/2160] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:28:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:28:11.204765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af2930-474d-4a88-9788-06496dd3383e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:11.432750 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af2930-474d-4a88-9788-06496dd3383e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:11.451741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a4590-f467-45f1-8e2e-dbd9b37c7e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:11.452686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a4590-f467-45f1-8e2e-dbd9b37c7e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:11.452956 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a4590-f467-45f1-8e2e-dbd9b37c7e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:11.453221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a4590-f467-45f1-8e2e-dbd9b37c7e58 admin admin] PATH_INFO: `/v3/users/b9f0d68834c74a55b77b99edb36a746d` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:11.454460 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a4590-f467-45f1-8e2e-dbd9b37c7e58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9f0d68834c74a55b77b99edb36a746d)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:11.457638 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a4590-f467-45f1-8e2e-dbd9b37c7e58 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:11.559719 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e33a4590-f467-45f1-8e2e-dbd9b37c7e58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9f0d68834c74a55b77b99edb36a746d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:11.561097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e33a4590-f467-45f1-8e2e-dbd9b37c7e58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9f0d68834c74a55b77b99edb36a746d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:11.564935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e33a4590-f467-45f1-8e2e-dbd9b37c7e58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9f0d68834c74a55b77b99edb36a746d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:11.605510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e33a4590-f467-45f1-8e2e-dbd9b37c7e58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9f0d68834c74a55b77b99edb36a746d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:11.616095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e33a4590-f467-45f1-8e2e-dbd9b37c7e58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9f0d68834c74a55b77b99edb36a746d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:11.657094 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1066/2161] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:28:11 2022] DELETE /identity/v3/users/b9f0d68834c74a55b77b99edb36a746d => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:28:11.661967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4015108-bf91-4da2-b158-cfbba34ff032 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:11.663012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4015108-bf91-4da2-b158-cfbba34ff032 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:11.663793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4015108-bf91-4da2-b158-cfbba34ff032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:11.734278 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c4015108-bf91-4da2-b158-cfbba34ff032 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:11.735501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c4015108-bf91-4da2-b158-cfbba34ff032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:11.837741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac79615-d813-4831-a668-9d01d4a2a861 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:11.838447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac79615-d813-4831-a668-9d01d4a2a861 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:11.839652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac79615-d813-4831-a668-9d01d4a2a861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:11.878599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1096/2162] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:11.909631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-7ac79615-d813-4831-a668-9d01d4a2a861 None None] MFA Rules not processed for user `a56f8a317a7a4942a9de8c6abf2da554`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:11.911590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-7ac79615-d813-4831-a668-9d01d4a2a861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:11.911735 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4015108-bf91-4da2-b158-cfbba34ff032 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:11.913440 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c4015108-bf91-4da2-b158-cfbba34ff032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:12.071487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1067/2163] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:28:12.090568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4015108-bf91-4da2-b158-cfbba34ff032 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:12.093626 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac79615-d813-4831-a668-9d01d4a2a861 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:12.098143 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-7ac79615-d813-4831-a668-9d01d4a2a861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:12.127509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b8432-ad61-4e84-ae21-06d2be0e5d99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:12.131288 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8432-ad61-4e84-ae21-06d2be0e5d99 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:12.131765 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8432-ad61-4e84-ae21-06d2be0e5d99 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:12.132216 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b8432-ad61-4e84-ae21-06d2be0e5d99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:12.151730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b8432-ad61-4e84-ae21-06d2be0e5d99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:12.153291 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b8432-ad61-4e84-ae21-06d2be0e5d99 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:12.177530 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1097/2164] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:28:12.204010 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac79615-d813-4831-a668-9d01d4a2a861 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:12.208673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ba18c-48ed-4dda-9e32-85646c2a97cc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '295bb46627e043989e6dba2805607c99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:12.210565 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ba18c-48ed-4dda-9e32-85646c2a97cc service glance] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:12.211075 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ba18c-48ed-4dda-9e32-85646c2a97cc service glance] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:12.211809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ba18c-48ed-4dda-9e32-85646c2a97cc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:12.213966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c05ba18c-48ed-4dda-9e32-85646c2a97cc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:12.294726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ba18c-48ed-4dda-9e32-85646c2a97cc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:12.301150 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ba18c-48ed-4dda-9e32-85646c2a97cc service glance] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:12.334600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1068/2165] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:28:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:28:12.568501 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7412f4-eb2f-4c18-a767-1779e6c77f77 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:12.568798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7412f4-eb2f-4c18-a767-1779e6c77f77 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:12.569097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7412f4-eb2f-4c18-a767-1779e6c77f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:12.624437 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-9f7412f4-eb2f-4c18-a767-1779e6c77f77 None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:12.625690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-9f7412f4-eb2f-4c18-a767-1779e6c77f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:12.843242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1098/2166] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:28:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:12.914662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ba18c-48ed-4dda-9e32-85646c2a97cc service glance] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:12.916367 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c05ba18c-48ed-4dda-9e32-85646c2a97cc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:13.091649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ba18c-48ed-4dda-9e32-85646c2a97cc service glance] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:13.104957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7412f4-eb2f-4c18-a767-1779e6c77f77 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:13.107464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-9f7412f4-eb2f-4c18-a767-1779e6c77f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:13.112192 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36116f84-a28a-4e58-a4da-59ae3f63bd2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:13.112877 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36116f84-a28a-4e58-a4da-59ae3f63bd2f service swift] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:13.113154 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36116f84-a28a-4e58-a4da-59ae3f63bd2f service swift] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:13.113442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36116f84-a28a-4e58-a4da-59ae3f63bd2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:13.135048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7412f4-eb2f-4c18-a767-1779e6c77f77 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:13.136041 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36116f84-a28a-4e58-a4da-59ae3f63bd2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:13.139110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36116f84-a28a-4e58-a4da-59ae3f63bd2f service swift] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:13.151930 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:13.153000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:13.153419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:13.153869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] PATH_INFO: `/v3/projects/310edc8624594bbb8d8e55f063e44688` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:13.157068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1069/2167] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:28:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:28:13.157693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=310edc8624594bbb8d8e55f063e44688)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:13.159696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:13.327240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '310edc8624594bbb8d8e55f063e44688', 'request_id': 'req-ffb12f46-f90b-4931-b337-8e8ea1d2f687'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:13.344810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '310edc8624594bbb8d8e55f063e44688', 'request_id': 'req-ffb12f46-f90b-4931-b337-8e8ea1d2f687'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:13.370453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '310edc8624594bbb8d8e55f063e44688', 'request_id': 'req-ffb12f46-f90b-4931-b337-8e8ea1d2f687'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:13.412724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] The token cache is being invalidate because project 310edc8624594bbb8d8e55f063e44688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:28:13.413020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:13.415080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:13.419631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1099/2168] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:28:13 2022] DELETE /identity/v3/projects/310edc8624594bbb8d8e55f063e44688 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:13.429824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36116f84-a28a-4e58-a4da-59ae3f63bd2f service swift] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:13.456281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36116f84-a28a-4e58-a4da-59ae3f63bd2f service swift] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:13.707370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134fd37-d41c-459a-856d-a9fa6aea5843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:13.709663 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134fd37-d41c-459a-856d-a9fa6aea5843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:13.710136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134fd37-d41c-459a-856d-a9fa6aea5843 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:13.710600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134fd37-d41c-459a-856d-a9fa6aea5843 admin admin] PATH_INFO: `/v3/users/6f23e45ec82c4862a96d13abba3a5245` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:13.712603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134fd37-d41c-459a-856d-a9fa6aea5843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f23e45ec82c4862a96d13abba3a5245)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:13.717656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134fd37-d41c-459a-856d-a9fa6aea5843 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:13.854546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c134fd37-d41c-459a-856d-a9fa6aea5843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f23e45ec82c4862a96d13abba3a5245'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:13.855939 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c134fd37-d41c-459a-856d-a9fa6aea5843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f23e45ec82c4862a96d13abba3a5245'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:13.857506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c134fd37-d41c-459a-856d-a9fa6aea5843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f23e45ec82c4862a96d13abba3a5245'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:13.898133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c134fd37-d41c-459a-856d-a9fa6aea5843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f23e45ec82c4862a96d13abba3a5245'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:13.932208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c134fd37-d41c-459a-856d-a9fa6aea5843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f23e45ec82c4862a96d13abba3a5245'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:13.965806 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1070/2169] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:28:13 2022] DELETE /identity/v3/users/6f23e45ec82c4862a96d13abba3a5245 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:15.582287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:15.584298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:15.607255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb12f46-f90b-4931-b337-8e8ea1d2f687 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:15.730380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af007df-1eb4-41a4-b04d-052bd138d4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:15.735308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af007df-1eb4-41a4-b04d-052bd138d4b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:15.735593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af007df-1eb4-41a4-b04d-052bd138d4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:15.736087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af007df-1eb4-41a4-b04d-052bd138d4b6 admin admin] PATH_INFO: `/v3/projects/3083351ddf8b4e94a3529ba2af7c5211` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:15.738061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af007df-1eb4-41a4-b04d-052bd138d4b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3083351ddf8b4e94a3529ba2af7c5211)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:15.744647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af007df-1eb4-41a4-b04d-052bd138d4b6 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:15.881958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6af007df-1eb4-41a4-b04d-052bd138d4b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3083351ddf8b4e94a3529ba2af7c5211', 'request_id': 'req-6af007df-1eb4-41a4-b04d-052bd138d4b6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:15.902908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6af007df-1eb4-41a4-b04d-052bd138d4b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3083351ddf8b4e94a3529ba2af7c5211', 'request_id': 'req-6af007df-1eb4-41a4-b04d-052bd138d4b6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:15.934850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6af007df-1eb4-41a4-b04d-052bd138d4b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3083351ddf8b4e94a3529ba2af7c5211', 'request_id': 'req-6af007df-1eb4-41a4-b04d-052bd138d4b6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:15.959074 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6af007df-1eb4-41a4-b04d-052bd138d4b6 admin admin] The token cache is being invalidate because project 3083351ddf8b4e94a3529ba2af7c5211 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:28:15.959398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6af007df-1eb4-41a4-b04d-052bd138d4b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:15.961051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6af007df-1eb4-41a4-b04d-052bd138d4b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:15.966987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1100/2170] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:28:15 2022] DELETE /identity/v3/projects/3083351ddf8b4e94a3529ba2af7c5211 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:15.968605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07a0cb-c2fe-4039-8275-c059de036765 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:15.969091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07a0cb-c2fe-4039-8275-c059de036765 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:15.969189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07a0cb-c2fe-4039-8275-c059de036765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:16.001411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-dc07a0cb-c2fe-4039-8275-c059de036765 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:16.004207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-dc07a0cb-c2fe-4039-8275-c059de036765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:16.170718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1071/2171] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:16.180679 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6b65c-a0ba-4f4a-8b98-d283a4c78b33 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:16.180970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6b65c-a0ba-4f4a-8b98-d283a4c78b33 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:16.181256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6b65c-a0ba-4f4a-8b98-d283a4c78b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:16.238224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e1b6b65c-a0ba-4f4a-8b98-d283a4c78b33 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:16.239847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e1b6b65c-a0ba-4f4a-8b98-d283a4c78b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:16.265630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1101/2172] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:16.265841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07a0cb-c2fe-4039-8275-c059de036765 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:16.282090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07a0cb-c2fe-4039-8275-c059de036765 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:16.289594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0641b-d136-4164-ac54-e50d4769c448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:16.290566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0641b-d136-4164-ac54-e50d4769c448 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:16.291550 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0641b-d136-4164-ac54-e50d4769c448 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:16.292571 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0641b-d136-4164-ac54-e50d4769c448 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:16.293736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0641b-d136-4164-ac54-e50d4769c448 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:28:16.294200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0641b-d136-4164-ac54-e50d4769c448 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:16.296736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0641b-d136-4164-ac54-e50d4769c448 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:16.318642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6b65c-a0ba-4f4a-8b98-d283a4c78b33 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:16.324395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1072/2173] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:28:16.336690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6b65c-a0ba-4f4a-8b98-d283a4c78b33 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:16.349879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c356928-6291-4fb9-bda4-b39a630b1235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:16.352119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c356928-6291-4fb9-bda4-b39a630b1235 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:16.352636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c356928-6291-4fb9-bda4-b39a630b1235 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:16.353058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c356928-6291-4fb9-bda4-b39a630b1235 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:16.353999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c356928-6291-4fb9-bda4-b39a630b1235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:16.360813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c356928-6291-4fb9-bda4-b39a630b1235 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:16.443506 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1102/2174] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:28:16 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:28:16.448467 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0641b-d136-4164-ac54-e50d4769c448 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:16.462187 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0641b-d136-4164-ac54-e50d4769c448 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:16.618569 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e165d9-2559-4769-b1d0-1bfd188b69b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:16.619266 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e165d9-2559-4769-b1d0-1bfd188b69b4 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:16.619557 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e165d9-2559-4769-b1d0-1bfd188b69b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:16.619825 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e165d9-2559-4769-b1d0-1bfd188b69b4 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:16.620546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e165d9-2559-4769-b1d0-1bfd188b69b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:16.634051 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e165d9-2559-4769-b1d0-1bfd188b69b4 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:16.677766 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1073/2175] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:28:16 2022] POST /identity/v3/users => generated 290 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:28:16.678505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c356928-6291-4fb9-bda4-b39a630b1235 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:16.706763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c356928-6291-4fb9-bda4-b39a630b1235 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:16.742055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd85841-0e2b-476a-83a6-69be5643c883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:16.745927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd85841-0e2b-476a-83a6-69be5643c883 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:16.746455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd85841-0e2b-476a-83a6-69be5643c883 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:16.747580 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd85841-0e2b-476a-83a6-69be5643c883 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:16.750209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd85841-0e2b-476a-83a6-69be5643c883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:16.754526 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd85841-0e2b-476a-83a6-69be5643c883 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:16.791761 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1103/2176] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:28:16.796856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e165d9-2559-4769-b1d0-1bfd188b69b4 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:16.814561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e165d9-2559-4769-b1d0-1bfd188b69b4 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:16.829534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c5d7b-8ad5-4266-b657-9f48e81aa1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:16.830223 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c5d7b-8ad5-4266-b657-9f48e81aa1e4 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:16.830505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c5d7b-8ad5-4266-b657-9f48e81aa1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:16.830780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c5d7b-8ad5-4266-b657-9f48e81aa1e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:16.831508 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c5d7b-8ad5-4266-b657-9f48e81aa1e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:16.835024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c5d7b-8ad5-4266-b657-9f48e81aa1e4 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:16.868944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd85841-0e2b-476a-83a6-69be5643c883 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:16.877606 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1074/2177] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:28:16.890853 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd85841-0e2b-476a-83a6-69be5643c883 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:16.910509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ecca9-37a0-4da3-b911-e09b029efc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:16.915864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ecca9-37a0-4da3-b911-e09b029efc29 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:16.916382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ecca9-37a0-4da3-b911-e09b029efc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:16.916839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ecca9-37a0-4da3-b911-e09b029efc29 admin admin] PATH_INFO: `/v3/projects/00bf7594548d41bb9ec8ac3200af4255/users/b01409779eaf460fa3fcbed0c1270b2c/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:17.021324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ecca9-37a0-4da3-b911-e09b029efc29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00bf7594548d41bb9ec8ac3200af4255, user_id=b01409779eaf460fa3fcbed0c1270b2c, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:17.039623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ecca9-37a0-4da3-b911-e09b029efc29 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:17.113584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1104/2178] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:28:16 2022] PUT /identity/v3/projects/00bf7594548d41bb9ec8ac3200af4255/users/b01409779eaf460fa3fcbed0c1270b2c/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:17.125716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924923e4-094d-4aee-a3ce-32136e6483d4 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:17.126294 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924923e4-094d-4aee-a3ce-32136e6483d4 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:17.126294 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924923e4-094d-4aee-a3ce-32136e6483d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:17.238608 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-924923e4-094d-4aee-a3ce-32136e6483d4 None None] MFA Rules not processed for user `b01409779eaf460fa3fcbed0c1270b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:17.240574 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-924923e4-094d-4aee-a3ce-32136e6483d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:17.805785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1075/2179] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 681 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:17.810713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ecca9-37a0-4da3-b911-e09b029efc29 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:17.957271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ecca9-37a0-4da3-b911-e09b029efc29 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:17.979728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957c4c9-b2bb-42d2-8d2f-c0c0804aaf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:17.980725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957c4c9-b2bb-42d2-8d2f-c0c0804aaf1e admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:17.981164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957c4c9-b2bb-42d2-8d2f-c0c0804aaf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:17.981601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957c4c9-b2bb-42d2-8d2f-c0c0804aaf1e admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:17.982452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957c4c9-b2bb-42d2-8d2f-c0c0804aaf1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:17.985142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957c4c9-b2bb-42d2-8d2f-c0c0804aaf1e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:18.037856 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1105/2180] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:28:17 2022] POST /identity/v3/projects => generated 379 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:28:18.042236 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924923e4-094d-4aee-a3ce-32136e6483d4 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:18.056151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924923e4-094d-4aee-a3ce-32136e6483d4 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:18.306979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cd9cf-fbfd-4697-b0bc-1d04e052e37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:18.307687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cd9cf-fbfd-4697-b0bc-1d04e052e37b admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:18.307956 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cd9cf-fbfd-4697-b0bc-1d04e052e37b admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:18.308233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cd9cf-fbfd-4697-b0bc-1d04e052e37b admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:18.308945 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524cd9cf-fbfd-4697-b0bc-1d04e052e37b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:18.310987 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524cd9cf-fbfd-4697-b0bc-1d04e052e37b admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:18.375542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1076/2181] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:28:18 2022] POST /identity/v3/users => generated 296 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:28:18.379424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957c4c9-b2bb-42d2-8d2f-c0c0804aaf1e admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:18.413810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957c4c9-b2bb-42d2-8d2f-c0c0804aaf1e admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:18.465011 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551c590-47ab-409a-b4ad-3ddfaf3c8b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:18.467727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551c590-47ab-409a-b4ad-3ddfaf3c8b10 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:18.471208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551c590-47ab-409a-b4ad-3ddfaf3c8b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:18.471753 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551c590-47ab-409a-b4ad-3ddfaf3c8b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:18.475412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551c590-47ab-409a-b4ad-3ddfaf3c8b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:18.478185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551c590-47ab-409a-b4ad-3ddfaf3c8b10 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:18.526221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1106/2182] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:28:18.529794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cd9cf-fbfd-4697-b0bc-1d04e052e37b admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:18.550747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cd9cf-fbfd-4697-b0bc-1d04e052e37b admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:18.564692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337e4f2-eba8-415a-b695-72c60aa41d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:18.567272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337e4f2-eba8-415a-b695-72c60aa41d3a admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:18.568435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337e4f2-eba8-415a-b695-72c60aa41d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:18.571692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337e4f2-eba8-415a-b695-72c60aa41d3a admin admin] PATH_INFO: `/v3/projects/db9cc4855d6b495186cb8fb40c8f40e4/users/c4b7a5de0bb840ce9865b16668913faf/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:18.649371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e337e4f2-eba8-415a-b695-72c60aa41d3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=db9cc4855d6b495186cb8fb40c8f40e4, user_id=c4b7a5de0bb840ce9865b16668913faf, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:18.653255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e337e4f2-eba8-415a-b695-72c60aa41d3a admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:18.705672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1077/2183] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:28:18 2022] PUT /identity/v3/projects/db9cc4855d6b495186cb8fb40c8f40e4/users/c4b7a5de0bb840ce9865b16668913faf/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:28:18.706593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551c590-47ab-409a-b4ad-3ddfaf3c8b10 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:18.722958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551c590-47ab-409a-b4ad-3ddfaf3c8b10 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:18.834494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f115f-8f89-4b7e-b13a-4fb205e3e7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:18.840884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f115f-8f89-4b7e-b13a-4fb205e3e7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:18.842433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f115f-8f89-4b7e-b13a-4fb205e3e7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:18.843169 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f115f-8f89-4b7e-b13a-4fb205e3e7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:18.847165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f115f-8f89-4b7e-b13a-4fb205e3e7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:18.853845 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f115f-8f89-4b7e-b13a-4fb205e3e7b9 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:18.883998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1107/2184] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:28:18.889524 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337e4f2-eba8-415a-b695-72c60aa41d3a admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:18.906745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337e4f2-eba8-415a-b695-72c60aa41d3a admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:18.952429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aaa4f-4323-48da-8f9e-d745e16a2a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:18.955037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aaa4f-4323-48da-8f9e-d745e16a2a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:18.969197 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aaa4f-4323-48da-8f9e-d745e16a2a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:18.969794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aaa4f-4323-48da-8f9e-d745e16a2a55 admin admin] PATH_INFO: `/v3/projects/db9cc4855d6b495186cb8fb40c8f40e4/users/c4b7a5de0bb840ce9865b16668913faf/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:18.972686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aaa4f-4323-48da-8f9e-d745e16a2a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db9cc4855d6b495186cb8fb40c8f40e4, user_id=c4b7a5de0bb840ce9865b16668913faf, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:18.976800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aaa4f-4323-48da-8f9e-d745e16a2a55 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:19.029097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a718e92-e648-44f0-88fc-e0f4adf51b97 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:19.029745 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a718e92-e648-44f0-88fc-e0f4adf51b97 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:19.030274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a718e92-e648-44f0-88fc-e0f4adf51b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:19.032769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1078/2185] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:28:18 2022] PUT /identity/v3/projects/db9cc4855d6b495186cb8fb40c8f40e4/users/c4b7a5de0bb840ce9865b16668913faf/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:19.137839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-5a718e92-e648-44f0-88fc-e0f4adf51b97 None None] MFA Rules not processed for user `c4b7a5de0bb840ce9865b16668913faf`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:19.141765 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5a718e92-e648-44f0-88fc-e0f4adf51b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:19.358072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1108/2186] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:19.377561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aaa4f-4323-48da-8f9e-d745e16a2a55 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:19.379881 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-589aaa4f-4323-48da-8f9e-d745e16a2a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:19.496222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aaa4f-4323-48da-8f9e-d745e16a2a55 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:19.523494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e5ca4-7ac3-4b91-84c6-856c0e2dde6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:19.524449 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e5ca4-7ac3-4b91-84c6-856c0e2dde6d service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:19.524834 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e5ca4-7ac3-4b91-84c6-856c0e2dde6d service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:19.525198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e5ca4-7ac3-4b91-84c6-856c0e2dde6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:19.572520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e5ca4-7ac3-4b91-84c6-856c0e2dde6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:19.580174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e5ca4-7ac3-4b91-84c6-856c0e2dde6d service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:19.603393 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1079/2187] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:28:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:28:20.292221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a718e92-e648-44f0-88fc-e0f4adf51b97 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:20.401452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a718e92-e648-44f0-88fc-e0f4adf51b97 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:20.409045 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:20.413416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:20.414639 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:20.415097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] PATH_INFO: `/v3/users/b01409779eaf460fa3fcbed0c1270b2c` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:20.419453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b01409779eaf460fa3fcbed0c1270b2c)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:20.423842 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:20.546606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b01409779eaf460fa3fcbed0c1270b2c'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:20.548762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b01409779eaf460fa3fcbed0c1270b2c'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:20.581214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b01409779eaf460fa3fcbed0c1270b2c'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:20.596526 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b01409779eaf460fa3fcbed0c1270b2c'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:20.598059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b01409779eaf460fa3fcbed0c1270b2c'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:20.653482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1109/2188] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:28:20 2022] DELETE /identity/v3/users/b01409779eaf460fa3fcbed0c1270b2c => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:20.657801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073adf8-abfb-465a-9aad-d05ecf126cc2 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:20.658795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073adf8-abfb-465a-9aad-d05ecf126cc2 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:20.659299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073adf8-abfb-465a-9aad-d05ecf126cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:20.698460 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-b073adf8-abfb-465a-9aad-d05ecf126cc2 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:20.700462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b073adf8-abfb-465a-9aad-d05ecf126cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:20.929872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1080/2189] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:20.943031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:20.944671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:21.085699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d76839-a11c-4a59-bbb3-357a69ec3d8d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:21.090294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ba2b7-ce2d-410e-8e56-0244a1165252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:21.102760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ba2b7-ce2d-410e-8e56-0244a1165252 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:21.106451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ba2b7-ce2d-410e-8e56-0244a1165252 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:21.110734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ba2b7-ce2d-410e-8e56-0244a1165252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:21.147719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ba2b7-ce2d-410e-8e56-0244a1165252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:21.150427 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ba2b7-ce2d-410e-8e56-0244a1165252 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:21.187234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1110/2190] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:28:21.846219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073adf8-abfb-465a-9aad-d05ecf126cc2 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:21.849307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b073adf8-abfb-465a-9aad-d05ecf126cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:21.882548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073adf8-abfb-465a-9aad-d05ecf126cc2 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:21.906671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:21.907718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:21.907941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:21.908559 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] PATH_INFO: `/v3/projects/00bf7594548d41bb9ec8ac3200af4255` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:21.910106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00bf7594548d41bb9ec8ac3200af4255)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:21.915842 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:22.006361 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00bf7594548d41bb9ec8ac3200af4255', 'request_id': 'req-8b7d3483-ee2b-472c-b953-a59c721d2633'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:22.024196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00bf7594548d41bb9ec8ac3200af4255', 'request_id': 'req-8b7d3483-ee2b-472c-b953-a59c721d2633'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:22.039240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00bf7594548d41bb9ec8ac3200af4255', 'request_id': 'req-8b7d3483-ee2b-472c-b953-a59c721d2633'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:22.118138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] The token cache is being invalidate because project 00bf7594548d41bb9ec8ac3200af4255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:28:22.118587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:22.120172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:22.123814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1081/2191] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:28:21 2022] DELETE /identity/v3/projects/00bf7594548d41bb9ec8ac3200af4255 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:22.126975 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ba2b7-ce2d-410e-8e56-0244a1165252 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:22.141184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ba2b7-ce2d-410e-8e56-0244a1165252 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:22.239789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3ef75-6eaa-4c0b-b1c6-c9cb43bf93ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:22.241098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3ef75-6eaa-4c0b-b1c6-c9cb43bf93ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:22.241613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3ef75-6eaa-4c0b-b1c6-c9cb43bf93ee admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:22.242028 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3ef75-6eaa-4c0b-b1c6-c9cb43bf93ee admin admin] PATH_INFO: `/v3/users/c4b7a5de0bb840ce9865b16668913faf` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:22.243749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3ef75-6eaa-4c0b-b1c6-c9cb43bf93ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4b7a5de0bb840ce9865b16668913faf)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:22.246123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3ef75-6eaa-4c0b-b1c6-c9cb43bf93ee admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:22.348413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-86e3ef75-6eaa-4c0b-b1c6-c9cb43bf93ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4b7a5de0bb840ce9865b16668913faf'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:22.349585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-86e3ef75-6eaa-4c0b-b1c6-c9cb43bf93ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4b7a5de0bb840ce9865b16668913faf'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:22.360860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-86e3ef75-6eaa-4c0b-b1c6-c9cb43bf93ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4b7a5de0bb840ce9865b16668913faf'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:22.373002 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-86e3ef75-6eaa-4c0b-b1c6-c9cb43bf93ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4b7a5de0bb840ce9865b16668913faf'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:22.373781 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-86e3ef75-6eaa-4c0b-b1c6-c9cb43bf93ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4b7a5de0bb840ce9865b16668913faf'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:22.408240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1111/2192] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:28:22 2022] DELETE /identity/v3/users/c4b7a5de0bb840ce9865b16668913faf => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:22.909603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:22.914819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:22.942328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d3483-ee2b-472c-b953-a59c721d2633 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:23.102047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:23.103744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:23.104402 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:23.104831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a admin admin] PATH_INFO: `/v3/projects/db9cc4855d6b495186cb8fb40c8f40e4` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:23.107700 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a admin admin] RBAC: Authorizing `identity:delete_project(project_id=db9cc4855d6b495186cb8fb40c8f40e4)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:23.110263 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:23.210739 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db9cc4855d6b495186cb8fb40c8f40e4', 'request_id': 'req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:23.226455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db9cc4855d6b495186cb8fb40c8f40e4', 'request_id': 'req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:23.236650 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db9cc4855d6b495186cb8fb40c8f40e4', 'request_id': 'req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:23.266791 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a admin admin] The token cache is being invalidate because project db9cc4855d6b495186cb8fb40c8f40e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:28:23.267093 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:23.269000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-805c7237-f0fb-4531-8e11-7e1aa1c5a74a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:23.271952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1082/2193] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:28:22 2022] DELETE /identity/v3/projects/db9cc4855d6b495186cb8fb40c8f40e4 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:23.276782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b257d57-cf0e-4b82-ad5c-206f51013199 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:23.277952 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b257d57-cf0e-4b82-ad5c-206f51013199 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:23.278312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b257d57-cf0e-4b82-ad5c-206f51013199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:23.336173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-7b257d57-cf0e-4b82-ad5c-206f51013199 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:23.337654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-7b257d57-cf0e-4b82-ad5c-206f51013199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:23.470092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1112/2194] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:23.477673 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715b82e-5d95-4c7d-94bb-d2112584a763 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:23.477969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715b82e-5d95-4c7d-94bb-d2112584a763 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:23.478255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715b82e-5d95-4c7d-94bb-d2112584a763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:23.510219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-e715b82e-5d95-4c7d-94bb-d2112584a763 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:23.515055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e715b82e-5d95-4c7d-94bb-d2112584a763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:23.530080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1083/2195] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:23.531966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b257d57-cf0e-4b82-ad5c-206f51013199 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:23.542006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b257d57-cf0e-4b82-ad5c-206f51013199 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:23.552662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90d26f-5272-4eaf-852c-55e8157a6a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:23.553441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90d26f-5272-4eaf-852c-55e8157a6a27 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:23.553868 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90d26f-5272-4eaf-852c-55e8157a6a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:23.554325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90d26f-5272-4eaf-852c-55e8157a6a27 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:23.555171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90d26f-5272-4eaf-852c-55e8157a6a27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:28:23.555614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90d26f-5272-4eaf-852c-55e8157a6a27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:23.558294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90d26f-5272-4eaf-852c-55e8157a6a27 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:23.584191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715b82e-5d95-4c7d-94bb-d2112584a763 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:23.593044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1113/2196] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:28:23.596770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715b82e-5d95-4c7d-94bb-d2112584a763 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:23.608889 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9244fc-1807-49a0-9cb1-023a8fc80754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:23.610325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9244fc-1807-49a0-9cb1-023a8fc80754 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:23.612330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9244fc-1807-49a0-9cb1-023a8fc80754 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:23.612674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9244fc-1807-49a0-9cb1-023a8fc80754 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:23.613466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9244fc-1807-49a0-9cb1-023a8fc80754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:23.630738 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9244fc-1807-49a0-9cb1-023a8fc80754 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:23.698986 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1084/2197] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:28:23 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:28:23.701459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90d26f-5272-4eaf-852c-55e8157a6a27 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:23.737378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90d26f-5272-4eaf-852c-55e8157a6a27 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:23.935489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a160e2d-ac89-4580-b74d-906146162a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:23.936232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a160e2d-ac89-4580-b74d-906146162a73 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:23.936601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a160e2d-ac89-4580-b74d-906146162a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:23.936913 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a160e2d-ac89-4580-b74d-906146162a73 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:23.937771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a160e2d-ac89-4580-b74d-906146162a73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:23.939871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a160e2d-ac89-4580-b74d-906146162a73 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:23.988898 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1114/2198] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:28:23 2022] POST /identity/v3/users => generated 288 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:28:23.997425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9244fc-1807-49a0-9cb1-023a8fc80754 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:24.011317 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9244fc-1807-49a0-9cb1-023a8fc80754 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:24.031077 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d13969-91c2-4aaf-be9f-d735d9cba084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:24.031785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d13969-91c2-4aaf-be9f-d735d9cba084 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:24.032100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d13969-91c2-4aaf-be9f-d735d9cba084 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:24.032381 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d13969-91c2-4aaf-be9f-d735d9cba084 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:24.033123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d13969-91c2-4aaf-be9f-d735d9cba084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:24.034741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d13969-91c2-4aaf-be9f-d735d9cba084 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:24.063408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1085/2199] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:28:24.069292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a160e2d-ac89-4580-b74d-906146162a73 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:24.096638 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a160e2d-ac89-4580-b74d-906146162a73 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:24.103176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4e86d-0bd2-4202-ac75-f5eb7b32ef3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:24.104177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4e86d-0bd2-4202-ac75-f5eb7b32ef3a admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:24.104467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4e86d-0bd2-4202-ac75-f5eb7b32ef3a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:24.104830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4e86d-0bd2-4202-ac75-f5eb7b32ef3a admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:24.105786 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4e86d-0bd2-4202-ac75-f5eb7b32ef3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:24.107427 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4e86d-0bd2-4202-ac75-f5eb7b32ef3a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:24.128272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d13969-91c2-4aaf-be9f-d735d9cba084 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:24.149894 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1115/2200] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:28:24.152862 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d13969-91c2-4aaf-be9f-d735d9cba084 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:24.161233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad734e-772e-448a-a94e-42148d238a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:24.164544 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad734e-772e-448a-a94e-42148d238a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:24.165340 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad734e-772e-448a-a94e-42148d238a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:24.165960 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad734e-772e-448a-a94e-42148d238a7b admin admin] PATH_INFO: `/v3/projects/bb5b5a0c04c04645a94ca898c5f56228/users/8c9b6c91ba074d34a6007f850d9b435d/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:24.185623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad734e-772e-448a-a94e-42148d238a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb5b5a0c04c04645a94ca898c5f56228, user_id=8c9b6c91ba074d34a6007f850d9b435d, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:24.189120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad734e-772e-448a-a94e-42148d238a7b admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:24.240882 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1086/2201] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:28:24 2022] PUT /identity/v3/projects/bb5b5a0c04c04645a94ca898c5f56228/users/8c9b6c91ba074d34a6007f850d9b435d/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:24.244159 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de5c88-6edb-4fd6-8510-9f21c20b3c29 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:24.244964 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de5c88-6edb-4fd6-8510-9f21c20b3c29 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:24.245452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de5c88-6edb-4fd6-8510-9f21c20b3c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:24.288441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-f7de5c88-6edb-4fd6-8510-9f21c20b3c29 None None] MFA Rules not processed for user `8c9b6c91ba074d34a6007f850d9b435d`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:24.290565 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f7de5c88-6edb-4fd6-8510-9f21c20b3c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:24.433805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1116/2202] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:24.451581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad734e-772e-448a-a94e-42148d238a7b admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:24.453725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-87ad734e-772e-448a-a94e-42148d238a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:24.542565 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad734e-772e-448a-a94e-42148d238a7b admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:24.550997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed12488-1210-476a-83b3-8f47142b2d42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:24.550997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed12488-1210-476a-83b3-8f47142b2d42 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:24.551366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed12488-1210-476a-83b3-8f47142b2d42 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:24.551615 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed12488-1210-476a-83b3-8f47142b2d42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:24.566428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed12488-1210-476a-83b3-8f47142b2d42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:24.568781 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed12488-1210-476a-83b3-8f47142b2d42 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:24.582479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1087/2203] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:28:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:28:28.969321 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de5c88-6edb-4fd6-8510-9f21c20b3c29 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:29.133033 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de5c88-6edb-4fd6-8510-9f21c20b3c29 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:29.138344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:29.139406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:29.139854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:29.140543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] PATH_INFO: `/v3/users/8c9b6c91ba074d34a6007f850d9b435d` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:29.141983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c9b6c91ba074d34a6007f850d9b435d)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:29.143922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:29.206108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c9b6c91ba074d34a6007f850d9b435d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:29.206783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c9b6c91ba074d34a6007f850d9b435d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:29.219888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c9b6c91ba074d34a6007f850d9b435d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:29.229032 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c9b6c91ba074d34a6007f850d9b435d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:29.229995 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c9b6c91ba074d34a6007f850d9b435d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:29.254234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1117/2204] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:28:28 2022] DELETE /identity/v3/users/8c9b6c91ba074d34a6007f850d9b435d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:29.263950 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167efdb2-a0c5-4b0a-9f6d-9132286ca72c None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:29.264947 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167efdb2-a0c5-4b0a-9f6d-9132286ca72c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:29.265387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167efdb2-a0c5-4b0a-9f6d-9132286ca72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:29.330490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-167efdb2-a0c5-4b0a-9f6d-9132286ca72c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:29.331999 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-167efdb2-a0c5-4b0a-9f6d-9132286ca72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:29.433730 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1088/2205] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:28:29.444216 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:29.445614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:29.580986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17332324-28a1-4978-b939-5d68499c2829 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:29.586070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d7884-5511-4e19-91e1-15e90698e636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:29.586986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d7884-5511-4e19-91e1-15e90698e636 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:29.587268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d7884-5511-4e19-91e1-15e90698e636 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:29.587539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d7884-5511-4e19-91e1-15e90698e636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:29.600422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d7884-5511-4e19-91e1-15e90698e636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:29.602020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d7884-5511-4e19-91e1-15e90698e636 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:29.617292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1118/2206] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:28:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:28:30.002681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167efdb2-a0c5-4b0a-9f6d-9132286ca72c None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:30.004103 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-167efdb2-a0c5-4b0a-9f6d-9132286ca72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:30.046867 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167efdb2-a0c5-4b0a-9f6d-9132286ca72c None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:30.064740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31182f27-cd2e-45f4-8f61-9539bb5f6bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:30.065640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31182f27-cd2e-45f4-8f61-9539bb5f6bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:30.066290 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31182f27-cd2e-45f4-8f61-9539bb5f6bad admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:30.066573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31182f27-cd2e-45f4-8f61-9539bb5f6bad admin admin] PATH_INFO: `/v3/projects/bb5b5a0c04c04645a94ca898c5f56228` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:30.068322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31182f27-cd2e-45f4-8f61-9539bb5f6bad admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb5b5a0c04c04645a94ca898c5f56228)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:30.074564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31182f27-cd2e-45f4-8f61-9539bb5f6bad admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:30.163913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-31182f27-cd2e-45f4-8f61-9539bb5f6bad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb5b5a0c04c04645a94ca898c5f56228', 'request_id': 'req-31182f27-cd2e-45f4-8f61-9539bb5f6bad'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:30.170165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-31182f27-cd2e-45f4-8f61-9539bb5f6bad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb5b5a0c04c04645a94ca898c5f56228', 'request_id': 'req-31182f27-cd2e-45f4-8f61-9539bb5f6bad'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:30.179309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-31182f27-cd2e-45f4-8f61-9539bb5f6bad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb5b5a0c04c04645a94ca898c5f56228', 'request_id': 'req-31182f27-cd2e-45f4-8f61-9539bb5f6bad'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:30.193832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-31182f27-cd2e-45f4-8f61-9539bb5f6bad admin admin] The token cache is being invalidate because project bb5b5a0c04c04645a94ca898c5f56228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:28:30.194117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-31182f27-cd2e-45f4-8f61-9539bb5f6bad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:30.197353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-31182f27-cd2e-45f4-8f61-9539bb5f6bad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:28:30.198981 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1089/2207] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:28:30 2022] DELETE /identity/v3/projects/bb5b5a0c04c04645a94ca898c5f56228 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:28:31.931228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f06d6b-6994-4f12-b594-900146f8ede1 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:31.931825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f06d6b-6994-4f12-b594-900146f8ede1 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:31.931825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f06d6b-6994-4f12-b594-900146f8ede1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:31.948923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-a9f06d6b-6994-4f12-b594-900146f8ede1 None None] MFA Rules not processed for user `cb34feb512764ec5b9ca25c30c3d7159`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:31.950121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a9f06d6b-6994-4f12-b594-900146f8ede1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:32.009198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1119/2208] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:32.581256 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26accb8e-d04a-4ac8-b19b-76b4a4c786f2 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:32.581805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26accb8e-d04a-4ac8-b19b-76b4a4c786f2 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:32.581910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26accb8e-d04a-4ac8-b19b-76b4a4c786f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:32.600166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-26accb8e-d04a-4ac8-b19b-76b4a4c786f2 None None] MFA Rules not processed for user `25e4c84bd2174040a20fa9c60ddb6b55`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:32.601468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-26accb8e-d04a-4ac8-b19b-76b4a4c786f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:32.659267 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1090/2209] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:28:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:35.219938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f80e0d-a097-4b1d-a19b-c254eb7a1b8f None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:35.220408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f80e0d-a097-4b1d-a19b-c254eb7a1b8f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:35.221101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f80e0d-a097-4b1d-a19b-c254eb7a1b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:35.263406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-a2f80e0d-a097-4b1d-a19b-c254eb7a1b8f None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:35.264658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a2f80e0d-a097-4b1d-a19b-c254eb7a1b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:35.389466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1120/2210] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:28:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:35.407268 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26accb8e-d04a-4ac8-b19b-76b4a4c786f2 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:35.408896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-26accb8e-d04a-4ac8-b19b-76b4a4c786f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:35.570695 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26accb8e-d04a-4ac8-b19b-76b4a4c786f2 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:35.579529 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4616a24-5939-48e6-9518-b422a4dca5a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:35.580687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4616a24-5939-48e6-9518-b422a4dca5a7 service swift] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:35.581858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4616a24-5939-48e6-9518-b422a4dca5a7 service swift] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:35.582586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4616a24-5939-48e6-9518-b422a4dca5a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:35.604295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4616a24-5939-48e6-9518-b422a4dca5a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:35.606386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4616a24-5939-48e6-9518-b422a4dca5a7 service swift] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:35.616987 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1091/2211] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:28:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:28:48.283327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f80e0d-a097-4b1d-a19b-c254eb7a1b8f None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:48.284906 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a2f80e0d-a097-4b1d-a19b-c254eb7a1b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:48.338554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f80e0d-a097-4b1d-a19b-c254eb7a1b8f None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:48.342059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a1c5e-1041-42d9-a1c3-21ab2c5b380b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:48.342690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a1c5e-1041-42d9-a1c3-21ab2c5b380b service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:48.342969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a1c5e-1041-42d9-a1c3-21ab2c5b380b service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:48.343242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a1c5e-1041-42d9-a1c3-21ab2c5b380b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:48.344959 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f09a1c5e-1041-42d9-a1c3-21ab2c5b380b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:48.402315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a1c5e-1041-42d9-a1c3-21ab2c5b380b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:48.403729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a1c5e-1041-42d9-a1c3-21ab2c5b380b service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:48.413447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1121/2212] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:28:48 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:28:52.751713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a998c4-acdf-49ba-a3ce-aaea80daa16a None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:52.752330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a998c4-acdf-49ba-a3ce-aaea80daa16a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:52.752330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a998c4-acdf-49ba-a3ce-aaea80daa16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:52.794330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-e3a998c4-acdf-49ba-a3ce-aaea80daa16a None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:28:52.795550 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e3a998c4-acdf-49ba-a3ce-aaea80daa16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:28:52.811670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1092/2213] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:28:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:28:52.825555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a1c5e-1041-42d9-a1c3-21ab2c5b380b service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:28:52.838368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a1c5e-1041-42d9-a1c3-21ab2c5b380b service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:28:52.843851 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179150ea-6bb6-4177-97ad-f38f80c53c43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:28:52.844550 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179150ea-6bb6-4177-97ad-f38f80c53c43 service swift] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:28:52.845325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179150ea-6bb6-4177-97ad-f38f80c53c43 service swift] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:28:52.845609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179150ea-6bb6-4177-97ad-f38f80c53c43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:28:52.854627 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179150ea-6bb6-4177-97ad-f38f80c53c43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:28:52.856253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179150ea-6bb6-4177-97ad-f38f80c53c43 service swift] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:28:52.881023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1122/2214] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:28:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:29:02.399235 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfbc85-2967-44fc-ab4b-f550be1fb72b None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:02.399872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfbc85-2967-44fc-ab4b-f550be1fb72b None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:02.399872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfbc85-2967-44fc-ab4b-f550be1fb72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:02.423358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-fadfbc85-2967-44fc-ab4b-f550be1fb72b None None] MFA Rules not processed for user `87b7c3ec77d4493dac92bb61d0b376be`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:29:02.424904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-fadfbc85-2967-44fc-ab4b-f550be1fb72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:29:02.517089 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1093/2215] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:29:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:29:04.918728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179150ea-6bb6-4177-97ad-f38f80c53c43 service swift] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:29:04.927591 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179150ea-6bb6-4177-97ad-f38f80c53c43 service swift] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:29:04.931141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba72d16-996d-46b1-8be3-3ad21c633a84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:29:04.931777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba72d16-996d-46b1-8be3-3ad21c633a84 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:04.932081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba72d16-996d-46b1-8be3-3ad21c633a84 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:04.932357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba72d16-996d-46b1-8be3-3ad21c633a84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:04.934063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-bba72d16-996d-46b1-8be3-3ad21c633a84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:29:04.996335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba72d16-996d-46b1-8be3-3ad21c633a84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:29:04.997755 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba72d16-996d-46b1-8be3-3ad21c633a84 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:29:05.007548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1123/2216] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:29:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:29:09.582896 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d3ce9-ba02-4a02-9032-1762c393c774 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:09.584369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d3ce9-ba02-4a02-9032-1762c393c774 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:09.584369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d3ce9-ba02-4a02-9032-1762c393c774 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:09.585873 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1094/2217] 10.222.0.44 () {54 vars in 921 bytes} [Mon May 9 04:29:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:29:09.592613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d7a40-2df6-472b-afe4-87f26ba097d7 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:09.593487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d7a40-2df6-472b-afe4-87f26ba097d7 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:09.593777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d7a40-2df6-472b-afe4-87f26ba097d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:09.655210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-f96d7a40-2df6-472b-afe4-87f26ba097d7 None None] MFA Rules not processed for user `faf6107735424ff7901614953e7df403`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:29:09.656923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f96d7a40-2df6-472b-afe4-87f26ba097d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:29:09.740801 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1124/2218] 10.222.0.44 () {58 vars in 1018 bytes} [Mon May 9 04:29:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:29:09.754036 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d3ce9-ba02-4a02-9032-1762c393c774 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:29:09.755966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-366d3ce9-ba02-4a02-9032-1762c393c774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:29:09.812007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d3ce9-ba02-4a02-9032-1762c393c774 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:29:09.815395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff7be2-4ce9-4d6e-a60f-12e8a9d2762c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '295bb46627e043989e6dba2805607c99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:29:09.816063 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff7be2-4ce9-4d6e-a60f-12e8a9d2762c service glance] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:09.816342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff7be2-4ce9-4d6e-a60f-12e8a9d2762c service glance] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:09.816610 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff7be2-4ce9-4d6e-a60f-12e8a9d2762c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:09.826324 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff7be2-4ce9-4d6e-a60f-12e8a9d2762c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:29:09.827741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff7be2-4ce9-4d6e-a60f-12e8a9d2762c service glance] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:29:09.838683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1095/2219] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:29:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:29:09.902294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25d18a-9f99-4ca4-9ff8-4827bc87dca1 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:09.902509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25d18a-9f99-4ca4-9ff8-4827bc87dca1 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:09.902786 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25d18a-9f99-4ca4-9ff8-4827bc87dca1 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:09.904207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1125/2220] 10.222.0.44 () {54 vars in 921 bytes} [Mon May 9 04:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:29:09.909441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b27c8-47e5-4ecb-ab8e-7cf3240efbd2 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:09.909763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b27c8-47e5-4ecb-ab8e-7cf3240efbd2 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:09.910085 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b27c8-47e5-4ecb-ab8e-7cf3240efbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:09.928064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-fe6b27c8-47e5-4ecb-ab8e-7cf3240efbd2 None None] MFA Rules not processed for user `0d852975a2e84925a45775a16cc52f32`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:29:09.929242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-fe6b27c8-47e5-4ecb-ab8e-7cf3240efbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:29:09.935513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1096/2221] 10.222.0.44 () {58 vars in 1018 bytes} [Mon May 9 04:29:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:29:09.947151 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25d18a-9f99-4ca4-9ff8-4827bc87dca1 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:29:09.957693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25d18a-9f99-4ca4-9ff8-4827bc87dca1 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:29:09.961947 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5a3fa-96b5-4e82-8aa8-3bb58071ee6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:29:09.962584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5a3fa-96b5-4e82-8aa8-3bb58071ee6f service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:09.962866 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5a3fa-96b5-4e82-8aa8-3bb58071ee6f service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:09.963141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5a3fa-96b5-4e82-8aa8-3bb58071ee6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:09.969948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5a3fa-96b5-4e82-8aa8-3bb58071ee6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:29:09.971369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5a3fa-96b5-4e82-8aa8-3bb58071ee6f service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:29:09.979398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1126/2222] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:29:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:29:10.061696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086137b9-cd73-48d9-8b84-81885c8a4c5f None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:10.062110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086137b9-cd73-48d9-8b84-81885c8a4c5f None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:10.062394 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086137b9-cd73-48d9-8b84-81885c8a4c5f None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:10.063746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1097/2223] 10.222.0.44 () {54 vars in 921 bytes} [Mon May 9 04:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:29:10.069312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885c0fd-7a14-400c-9859-1a16995b2935 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:10.069603 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885c0fd-7a14-400c-9859-1a16995b2935 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:10.069883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885c0fd-7a14-400c-9859-1a16995b2935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:10.088940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-5885c0fd-7a14-400c-9859-1a16995b2935 None None] MFA Rules not processed for user `faf6107735424ff7901614953e7df403`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:29:10.090225 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5885c0fd-7a14-400c-9859-1a16995b2935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:29:10.100394 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1127/2224] 10.222.0.44 () {58 vars in 1018 bytes} [Mon May 9 04:29:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:29:10.111029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086137b9-cd73-48d9-8b84-81885c8a4c5f None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:29:10.122110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086137b9-cd73-48d9-8b84-81885c8a4c5f None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:29:10.125668 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355a970-6ec2-444a-afa9-26b036b390b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:29:10.126276 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355a970-6ec2-444a-afa9-26b036b390b0 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:10.126554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355a970-6ec2-444a-afa9-26b036b390b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:10.126825 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355a970-6ec2-444a-afa9-26b036b390b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:10.136590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355a970-6ec2-444a-afa9-26b036b390b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:29:10.138016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355a970-6ec2-444a-afa9-26b036b390b0 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:29:10.149213 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1098/2225] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:29:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:29:12.563170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885c0fd-7a14-400c-9859-1a16995b2935 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:29:12.572861 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885c0fd-7a14-400c-9859-1a16995b2935 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:29:12.576486 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76636b33-c40e-46f4-af83-a4a3081abbd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:29:12.577126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636b33-c40e-46f4-af83-a4a3081abbd4 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:12.577408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636b33-c40e-46f4-af83-a4a3081abbd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:12.577684 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76636b33-c40e-46f4-af83-a4a3081abbd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:12.579467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-76636b33-c40e-46f4-af83-a4a3081abbd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:29:12.641981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76636b33-c40e-46f4-af83-a4a3081abbd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:29:12.643408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76636b33-c40e-46f4-af83-a4a3081abbd4 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:29:12.653818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1128/2226] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:29:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:29:15.205184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c83bd1-165a-4538-9c0c-360c8ed1b697 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:15.206878 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c83bd1-165a-4538-9c0c-360c8ed1b697 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:15.207378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c83bd1-165a-4538-9c0c-360c8ed1b697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:15.227693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-c3c83bd1-165a-4538-9c0c-360c8ed1b697 None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:29:15.228952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c3c83bd1-165a-4538-9c0c-360c8ed1b697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:29:15.245253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1099/2227] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:29:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:29:15.272653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76636b33-c40e-46f4-af83-a4a3081abbd4 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:29:15.288151 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76636b33-c40e-46f4-af83-a4a3081abbd4 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:29:15.294926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1df78d-2214-4d05-a6ed-33638b1b9bdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:29:15.295835 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1df78d-2214-4d05-a6ed-33638b1b9bdf service swift] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:15.296408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1df78d-2214-4d05-a6ed-33638b1b9bdf service swift] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:15.296906 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1df78d-2214-4d05-a6ed-33638b1b9bdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:15.307721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1df78d-2214-4d05-a6ed-33638b1b9bdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:29:15.309522 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1df78d-2214-4d05-a6ed-33638b1b9bdf service swift] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:29:15.319422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1129/2228] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:29:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:29:32.780330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c83bd1-165a-4538-9c0c-360c8ed1b697 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:29:32.781794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c3c83bd1-165a-4538-9c0c-360c8ed1b697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:29:32.894039 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c83bd1-165a-4538-9c0c-360c8ed1b697 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:29:32.900559 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c62883-3ffb-4677-b42f-6e966e732d6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd41e05b7ef954dd596c0134b65434dd6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:29:32.901216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c62883-3ffb-4677-b42f-6e966e732d6f service placement] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:32.901497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c62883-3ffb-4677-b42f-6e966e732d6f service placement] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:32.901867 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c62883-3ffb-4677-b42f-6e966e732d6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:32.905460 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-17c62883-3ffb-4677-b42f-6e966e732d6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:29:32.918458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c62883-3ffb-4677-b42f-6e966e732d6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:29:32.920145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c62883-3ffb-4677-b42f-6e966e732d6f service placement] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:29:32.932075 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1100/2229] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:29:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:29:42.331003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55340adb-20d4-4c28-88ba-82ab5a8d3a4c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:42.331584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55340adb-20d4-4c28-88ba-82ab5a8d3a4c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:42.331767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55340adb-20d4-4c28-88ba-82ab5a8d3a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:42.349438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-55340adb-20d4-4c28-88ba-82ab5a8d3a4c None None] MFA Rules not processed for user `c56a403eeee84720ba765e2d4367df74`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:29:42.350848 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-55340adb-20d4-4c28-88ba-82ab5a8d3a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:29:42.406051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1130/2230] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:29:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:29:44.459191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103c2e6-6ed3-49dc-b2d8-68a2f06690ec None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:44.459686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103c2e6-6ed3-49dc-b2d8-68a2f06690ec None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:44.459792 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103c2e6-6ed3-49dc-b2d8-68a2f06690ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:44.510776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-8103c2e6-6ed3-49dc-b2d8-68a2f06690ec None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:29:44.512086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8103c2e6-6ed3-49dc-b2d8-68a2f06690ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:29:44.529228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1101/2231] 10.222.0.44 () {58 vars in 967 bytes} [Mon May 9 04:29:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:29:44.542596 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55340adb-20d4-4c28-88ba-82ab5a8d3a4c None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:29:44.564033 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55340adb-20d4-4c28-88ba-82ab5a8d3a4c None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:29:44.568903 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9072cf-3b57-4d02-95d7-cbe6b25c5924 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:29:44.569556 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9072cf-3b57-4d02-95d7-cbe6b25c5924 service swift] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:29:44.569838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9072cf-3b57-4d02-95d7-cbe6b25c5924 service swift] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:29:44.570110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9072cf-3b57-4d02-95d7-cbe6b25c5924 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:29:44.579924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9072cf-3b57-4d02-95d7-cbe6b25c5924 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:29:44.581916 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9072cf-3b57-4d02-95d7-cbe6b25c5924 service swift] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:29:44.589998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1131/2232] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:29:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:30:44.364685 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983367d3-0b32-4cb6-a337-3969266b6e24 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:30:44.365212 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983367d3-0b32-4cb6-a337-3969266b6e24 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:30:44.365319 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983367d3-0b32-4cb6-a337-3969266b6e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:30:44.385433 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-983367d3-0b32-4cb6-a337-3969266b6e24 None None] MFA Rules not processed for user `4f8f6c77f898475e8acf1c4042c3928b`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:30:44.386659 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-983367d3-0b32-4cb6-a337-3969266b6e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:30:44.447783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1102/2233] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:30:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:30:46.649972 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22953b0-f0af-47dc-8016-37ad360607c6 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:30:46.650474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22953b0-f0af-47dc-8016-37ad360607c6 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:30:46.650762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22953b0-f0af-47dc-8016-37ad360607c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:30:46.668973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-f22953b0-f0af-47dc-8016-37ad360607c6 None None] MFA Rules not processed for user `87b7c3ec77d4493dac92bb61d0b376be`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:30:46.670154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f22953b0-f0af-47dc-8016-37ad360607c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:30:46.677924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1132/2234] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:30:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:31:02.939131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a49176-66bb-49df-855e-02bc3c3d94ad None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:02.939671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a49176-66bb-49df-855e-02bc3c3d94ad None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:02.939671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a49176-66bb-49df-855e-02bc3c3d94ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:02.964346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-97a49176-66bb-49df-855e-02bc3c3d94ad None None] MFA Rules not processed for user `c56a403eeee84720ba765e2d4367df74`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:31:02.965560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-97a49176-66bb-49df-855e-02bc3c3d94ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:02.973208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1103/2235] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:31:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:31:11.354253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c093300-3aab-4e68-a90e-f47ac3660bae None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:11.354790 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c093300-3aab-4e68-a90e-f47ac3660bae None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:11.354901 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c093300-3aab-4e68-a90e-f47ac3660bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:11.372981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-0c093300-3aab-4e68-a90e-f47ac3660bae None None] MFA Rules not processed for user `85429c28c994443eb30b98ece56c4c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:31:11.374170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-0c093300-3aab-4e68-a90e-f47ac3660bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:11.426619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1133/2236] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:31:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:31:15.141444 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a49176-66bb-49df-855e-02bc3c3d94ad None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:15.150747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a49176-66bb-49df-855e-02bc3c3d94ad None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:15.154399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18a043-b3aa-48da-81a2-1e461e7032c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:15.155034 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18a043-b3aa-48da-81a2-1e461e7032c4 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:15.155316 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18a043-b3aa-48da-81a2-1e461e7032c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:15.155590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18a043-b3aa-48da-81a2-1e461e7032c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:15.157355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-dd18a043-b3aa-48da-81a2-1e461e7032c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:15.170020 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18a043-b3aa-48da-81a2-1e461e7032c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:15.171409 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18a043-b3aa-48da-81a2-1e461e7032c4 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:15.182476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1104/2237] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:31:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:31:15.529597 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c093300-3aab-4e68-a90e-f47ac3660bae None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:15.531088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-0c093300-3aab-4e68-a90e-f47ac3660bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:15.543535 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c093300-3aab-4e68-a90e-f47ac3660bae None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:15.547087 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b86285-f95c-4c39-8f15-cc167ca48a71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:15.547734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b86285-f95c-4c39-8f15-cc167ca48a71 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:15.548041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b86285-f95c-4c39-8f15-cc167ca48a71 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:15.548315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b86285-f95c-4c39-8f15-cc167ca48a71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:15.550045 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-31b86285-f95c-4c39-8f15-cc167ca48a71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:15.564038 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b86285-f95c-4c39-8f15-cc167ca48a71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:15.565461 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b86285-f95c-4c39-8f15-cc167ca48a71 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:15.576690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1134/2238] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:31:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:31:21.446399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18a043-b3aa-48da-81a2-1e461e7032c4 service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:21.448021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-dd18a043-b3aa-48da-81a2-1e461e7032c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:21.461454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18a043-b3aa-48da-81a2-1e461e7032c4 service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:21.467068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888602d7-a639-4753-8af1-6a31762c0dc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:21.467702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888602d7-a639-4753-8af1-6a31762c0dc3 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:21.468015 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888602d7-a639-4753-8af1-6a31762c0dc3 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:21.468294 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888602d7-a639-4753-8af1-6a31762c0dc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:21.470016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-888602d7-a639-4753-8af1-6a31762c0dc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:21.480420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888602d7-a639-4753-8af1-6a31762c0dc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:21.482003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888602d7-a639-4753-8af1-6a31762c0dc3 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:21.490621 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1105/2239] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:31:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:31:30.984932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b86285-f95c-4c39-8f15-cc167ca48a71 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:30.993334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b86285-f95c-4c39-8f15-cc167ca48a71 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:30.997055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d415c90-141c-4061-81b1-ccaae9fead94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:30.997690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d415c90-141c-4061-81b1-ccaae9fead94 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:30.998027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d415c90-141c-4061-81b1-ccaae9fead94 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:30.998303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d415c90-141c-4061-81b1-ccaae9fead94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:31.000092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-9d415c90-141c-4061-81b1-ccaae9fead94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:31.009058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d415c90-141c-4061-81b1-ccaae9fead94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:31.010718 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d415c90-141c-4061-81b1-ccaae9fead94 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:31.018347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1135/2240] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:31:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:31:35.461162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130eef0d-b8be-44bf-8d3a-0b724d170ff7 None None] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:35.461729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130eef0d-b8be-44bf-8d3a-0b724d170ff7 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:35.461893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130eef0d-b8be-44bf-8d3a-0b724d170ff7 None None] PATH_INFO: `/` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:35.463809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1106/2241] 10.222.0.44 () {54 vars in 921 bytes} [Mon May 9 04:31:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:31:35.470254 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbe093-8665-4e7f-ab99-71b275ba4299 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:35.470741 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbe093-8665-4e7f-ab99-71b275ba4299 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:35.470846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbe093-8665-4e7f-ab99-71b275ba4299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:35.494823 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-09dbe093-8665-4e7f-ab99-71b275ba4299 None None] MFA Rules not processed for user `0d852975a2e84925a45775a16cc52f32`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:31:35.496048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-09dbe093-8665-4e7f-ab99-71b275ba4299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:35.502507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1136/2242] 10.222.0.44 () {58 vars in 1018 bytes} [Mon May 9 04:31:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:31:35.518657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130eef0d-b8be-44bf-8d3a-0b724d170ff7 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:35.529759 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130eef0d-b8be-44bf-8d3a-0b724d170ff7 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:35.533349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae4d8f-c2cb-4b8b-aad6-dc37434a7bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:35.533995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae4d8f-c2cb-4b8b-aad6-dc37434a7bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:35.534276 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae4d8f-c2cb-4b8b-aad6-dc37434a7bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:35.534551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae4d8f-c2cb-4b8b-aad6-dc37434a7bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:35.542421 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae4d8f-c2cb-4b8b-aad6-dc37434a7bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:35.543837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae4d8f-c2cb-4b8b-aad6-dc37434a7bd3 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:35.552171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1107/2243] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:31:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:31:35.651272 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9da81-2c97-4e2b-9867-2002f1fa03e3 None None] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:35.651561 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9da81-2c97-4e2b-9867-2002f1fa03e3 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:35.651838 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9da81-2c97-4e2b-9867-2002f1fa03e3 None None] PATH_INFO: `/` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:35.653299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1137/2244] 10.222.0.44 () {54 vars in 921 bytes} [Mon May 9 04:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 04:31:35.658875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfb353-4edd-48a7-95b1-9b3de28710c5 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:35.659168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfb353-4edd-48a7-95b1-9b3de28710c5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:35.659450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfb353-4edd-48a7-95b1-9b3de28710c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:35.677927 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-29bfb353-4edd-48a7-95b1-9b3de28710c5 None None] MFA Rules not processed for user `faf6107735424ff7901614953e7df403`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:31:35.679135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-29bfb353-4edd-48a7-95b1-9b3de28710c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:35.689031 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1108/2245] 10.222.0.44 () {58 vars in 1018 bytes} [Mon May 9 04:31:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:31:35.699934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9da81-2c97-4e2b-9867-2002f1fa03e3 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:35.710528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9da81-2c97-4e2b-9867-2002f1fa03e3 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:35.714943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142ac9a-dcb5-474b-8421-e840496c88b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:35.715603 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142ac9a-dcb5-474b-8421-e840496c88b5 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:35.715886 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142ac9a-dcb5-474b-8421-e840496c88b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:35.716184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142ac9a-dcb5-474b-8421-e840496c88b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:35.725905 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142ac9a-dcb5-474b-8421-e840496c88b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:35.727315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142ac9a-dcb5-474b-8421-e840496c88b5 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:35.737843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1138/2246] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:31:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:31:35.800592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfb353-4edd-48a7-95b1-9b3de28710c5 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:35.811734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfb353-4edd-48a7-95b1-9b3de28710c5 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:35.816886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937e1e8-dc30-4eba-84fb-cb1f80e0c47a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd41e05b7ef954dd596c0134b65434dd6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:35.817528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937e1e8-dc30-4eba-84fb-cb1f80e0c47a service placement] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:35.817809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937e1e8-dc30-4eba-84fb-cb1f80e0c47a service placement] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:35.818083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937e1e8-dc30-4eba-84fb-cb1f80e0c47a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:35.820171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1937e1e8-dc30-4eba-84fb-cb1f80e0c47a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:35.832179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937e1e8-dc30-4eba-84fb-cb1f80e0c47a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:35.833738 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937e1e8-dc30-4eba-84fb-cb1f80e0c47a service placement] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:35.844168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1109/2247] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:31:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:31:36.115096 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142ac9a-dcb5-474b-8421-e840496c88b5 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:36.116603 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6142ac9a-dcb5-474b-8421-e840496c88b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:36.131789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142ac9a-dcb5-474b-8421-e840496c88b5 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:36.137948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f7c37-1f7d-4323-b2c5-88df323248de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:36.138584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f7c37-1f7d-4323-b2c5-88df323248de service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:36.138864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f7c37-1f7d-4323-b2c5-88df323248de service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:36.139137 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f7c37-1f7d-4323-b2c5-88df323248de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:36.140862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5a5f7c37-1f7d-4323-b2c5-88df323248de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:36.150926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f7c37-1f7d-4323-b2c5-88df323248de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:36.152518 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f7c37-1f7d-4323-b2c5-88df323248de service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:36.160993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1139/2248] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:31:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:31:42.379021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937e1e8-dc30-4eba-84fb-cb1f80e0c47a service placement] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:42.409448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937e1e8-dc30-4eba-84fb-cb1f80e0c47a service placement] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:42.417862 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218de5d-caca-4544-8a41-5c5c9f083cb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:42.418741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218de5d-caca-4544-8a41-5c5c9f083cb9 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:42.422956 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218de5d-caca-4544-8a41-5c5c9f083cb9 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:42.423465 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218de5d-caca-4544-8a41-5c5c9f083cb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:42.425717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f218de5d-caca-4544-8a41-5c5c9f083cb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:42.443157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218de5d-caca-4544-8a41-5c5c9f083cb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:42.445217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218de5d-caca-4544-8a41-5c5c9f083cb9 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:42.456347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1110/2249] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:31:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:31:45.234704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb95c06-89d3-479f-bfbe-c109540262b8 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:45.235211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb95c06-89d3-479f-bfbe-c109540262b8 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:45.235416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb95c06-89d3-479f-bfbe-c109540262b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:45.255601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2fb95c06-89d3-479f-bfbe-c109540262b8 None None] MFA Rules not processed for user `cb34feb512764ec5b9ca25c30c3d7159`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:31:45.256823 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2fb95c06-89d3-479f-bfbe-c109540262b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:45.263693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1140/2250] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:31:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:31:45.601810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218de5d-caca-4544-8a41-5c5c9f083cb9 service nova] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:45.611282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218de5d-caca-4544-8a41-5c5c9f083cb9 service nova] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:45.614707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c723c4-98b9-4738-95b8-f2c8f3bc09df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:45.615338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c723c4-98b9-4738-95b8-f2c8f3bc09df service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:45.615617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c723c4-98b9-4738-95b8-f2c8f3bc09df service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:45.615902 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c723c4-98b9-4738-95b8-f2c8f3bc09df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:45.617616 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b9c723c4-98b9-4738-95b8-f2c8f3bc09df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:45.626814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c723c4-98b9-4738-95b8-f2c8f3bc09df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:45.628513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c723c4-98b9-4738-95b8-f2c8f3bc09df service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:45.636577 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1111/2251] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:31:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:31:47.055432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a6aa-9342-4ab8-bcc3-1935b88eb458 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:47.055992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a6aa-9342-4ab8-bcc3-1935b88eb458 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:47.056137 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a6aa-9342-4ab8-bcc3-1935b88eb458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:47.074791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-fd46a6aa-9342-4ab8-bcc3-1935b88eb458 None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:31:47.076012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-fd46a6aa-9342-4ab8-bcc3-1935b88eb458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:47.083872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1141/2252] 10.222.0.44 () {58 vars in 967 bytes} [Mon May 9 04:31:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:31:47.136593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c723c4-98b9-4738-95b8-f2c8f3bc09df service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:47.151375 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c723c4-98b9-4738-95b8-f2c8f3bc09df service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:47.157358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82921bda-36e7-4189-a296-ddc0a5a66d89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:47.158295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82921bda-36e7-4189-a296-ddc0a5a66d89 service swift] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:47.158583 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82921bda-36e7-4189-a296-ddc0a5a66d89 service swift] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:47.159030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82921bda-36e7-4189-a296-ddc0a5a66d89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:47.168733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82921bda-36e7-4189-a296-ddc0a5a66d89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:47.170557 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82921bda-36e7-4189-a296-ddc0a5a66d89 service swift] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:47.180728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1112/2253] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:31:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:31:47.733475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46a6aa-9342-4ab8-bcc3-1935b88eb458 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:47.750088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46a6aa-9342-4ab8-bcc3-1935b88eb458 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:47.780951 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd0ab3-e887-4256-b916-871e1931d6e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:47.781636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd0ab3-e887-4256-b916-871e1931d6e4 service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:47.781920 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd0ab3-e887-4256-b916-871e1931d6e4 service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:47.782201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd0ab3-e887-4256-b916-871e1931d6e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:47.784908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-05bd0ab3-e887-4256-b916-871e1931d6e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:47.826584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd0ab3-e887-4256-b916-871e1931d6e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:47.828214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd0ab3-e887-4256-b916-871e1931d6e4 service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:47.846418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1142/2254] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:31:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:31:56.822943 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82921bda-36e7-4189-a296-ddc0a5a66d89 service swift] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:56.824751 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-82921bda-36e7-4189-a296-ddc0a5a66d89 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:56.908830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82921bda-36e7-4189-a296-ddc0a5a66d89 service swift] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:56.913388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:56.914253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:56.914539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:56.914830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] PATH_INFO: `/v3/users/25e4c84bd2174040a20fa9c60ddb6b55` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:56.916641 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] RBAC: Authorizing `identity:delete_user(user_id=25e4c84bd2174040a20fa9c60ddb6b55)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:56.918536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:56.967243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25e4c84bd2174040a20fa9c60ddb6b55'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:56.967980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25e4c84bd2174040a20fa9c60ddb6b55'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:56.968667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25e4c84bd2174040a20fa9c60ddb6b55'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:56.976523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25e4c84bd2174040a20fa9c60ddb6b55'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:56.983278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25e4c84bd2174040a20fa9c60ddb6b55'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.001282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1113/2255] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:31:56 2022] DELETE /identity/v3/users/25e4c84bd2174040a20fa9c60ddb6b55 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:31:57.252429 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd0ab3-e887-4256-b916-871e1931d6e4 service nova] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:57.264017 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd0ab3-e887-4256-b916-871e1931d6e4 service nova] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:57.322850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:57.323711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:57.324022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:57.324294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] PATH_INFO: `/v3/projects/cf2a64ab3bc1448b8a934e2d103bb161` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:57.325468 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf2a64ab3bc1448b8a934e2d103bb161)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:57.327357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:57.374574 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf2a64ab3bc1448b8a934e2d103bb161', 'request_id': 'req-6d01ebb1-05b4-4284-9d50-208684e5d5cb'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.379819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf2a64ab3bc1448b8a934e2d103bb161', 'request_id': 'req-6d01ebb1-05b4-4284-9d50-208684e5d5cb'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.387656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf2a64ab3bc1448b8a934e2d103bb161', 'request_id': 'req-6d01ebb1-05b4-4284-9d50-208684e5d5cb'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.398872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] The token cache is being invalidate because project cf2a64ab3bc1448b8a934e2d103bb161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:31:57.399168 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.401512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.402253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1143/2256] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:31:57 2022] DELETE /identity/v3/projects/cf2a64ab3bc1448b8a934e2d103bb161 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:31:57.406286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:57.407710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:57.420972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599127b7-3670-4cbe-90c8-b0af092b30df admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:57.476474 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:57.477342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:57.477732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:57.478048 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] PATH_INFO: `/v3/users/a56f8a317a7a4942a9de8c6abf2da554` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:57.479221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a56f8a317a7a4942a9de8c6abf2da554)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:57.481177 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:57.526868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a56f8a317a7a4942a9de8c6abf2da554'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.527451 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a56f8a317a7a4942a9de8c6abf2da554'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.528098 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a56f8a317a7a4942a9de8c6abf2da554'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.535195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a56f8a317a7a4942a9de8c6abf2da554'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.542153 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a56f8a317a7a4942a9de8c6abf2da554'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.558299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1114/2257] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:31:57 2022] DELETE /identity/v3/users/a56f8a317a7a4942a9de8c6abf2da554 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:31:57.728859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:31:57.730356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:31:57.743764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01ebb1-05b4-4284-9d50-208684e5d5cb admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:31:57.800519 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:31:57.801305 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:31:57.801584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:31:57.801854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] PATH_INFO: `/v3/projects/678b52347f054baa98342f098b23a741` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:31:57.803054 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=678b52347f054baa98342f098b23a741)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:31:57.804994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:31:57.850746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '678b52347f054baa98342f098b23a741', 'request_id': 'req-27aff431-d933-4a7b-aa46-039580e64af7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.855779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '678b52347f054baa98342f098b23a741', 'request_id': 'req-27aff431-d933-4a7b-aa46-039580e64af7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.863908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '678b52347f054baa98342f098b23a741', 'request_id': 'req-27aff431-d933-4a7b-aa46-039580e64af7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.875665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] The token cache is being invalidate because project 678b52347f054baa98342f098b23a741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:31:57.875980 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.876784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:31:57.878357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1144/2258] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:31:57 2022] DELETE /identity/v3/projects/678b52347f054baa98342f098b23a741 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:02.416804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:02.418237 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:02.476190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fcdb0-d29b-4129-b665-202491ba9d94 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:02.523489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f3048-1aa6-4712-954c-2c94f7c5e75f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:02.524108 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f3048-1aa6-4712-954c-2c94f7c5e75f service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:02.524384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f3048-1aa6-4712-954c-2c94f7c5e75f service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:02.524655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f3048-1aa6-4712-954c-2c94f7c5e75f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:02.526549 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-578f3048-1aa6-4712-954c-2c94f7c5e75f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:02.584832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f3048-1aa6-4712-954c-2c94f7c5e75f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:02.586253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f3048-1aa6-4712-954c-2c94f7c5e75f service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:02.596087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1115/2259] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:32:04.641163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:04.643489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:04.725862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aff431-d933-4a7b-aa46-039580e64af7 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:04.732414 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d4387-cc7b-4d29-aad2-5da06455cbdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:04.733140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d4387-cc7b-4d29-aad2-5da06455cbdd service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:04.733430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d4387-cc7b-4d29-aad2-5da06455cbdd service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:04.733749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d4387-cc7b-4d29-aad2-5da06455cbdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:04.735864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-897d4387-cc7b-4d29-aad2-5da06455cbdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:04.757970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d4387-cc7b-4d29-aad2-5da06455cbdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:04.757970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d4387-cc7b-4d29-aad2-5da06455cbdd service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:04.771794 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1145/2260] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:32:06.292304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f3048-1aa6-4712-954c-2c94f7c5e75f service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:06.294223 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-578f3048-1aa6-4712-954c-2c94f7c5e75f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:06.305954 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f3048-1aa6-4712-954c-2c94f7c5e75f service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:06.312815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40472b29-28d9-465f-a704-5deae14f0daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:06.313698 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40472b29-28d9-465f-a704-5deae14f0daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:06.313977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40472b29-28d9-465f-a704-5deae14f0daa admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:06.314251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40472b29-28d9-465f-a704-5deae14f0daa admin admin] PATH_INFO: `/v3/users/87b7c3ec77d4493dac92bb61d0b376be` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:06.315413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40472b29-28d9-465f-a704-5deae14f0daa admin admin] RBAC: Authorizing `identity:delete_user(user_id=87b7c3ec77d4493dac92bb61d0b376be)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:06.317340 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40472b29-28d9-465f-a704-5deae14f0daa admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:06.363109 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-40472b29-28d9-465f-a704-5deae14f0daa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87b7c3ec77d4493dac92bb61d0b376be'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:06.363738 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-40472b29-28d9-465f-a704-5deae14f0daa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87b7c3ec77d4493dac92bb61d0b376be'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:06.364373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-40472b29-28d9-465f-a704-5deae14f0daa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87b7c3ec77d4493dac92bb61d0b376be'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:06.371414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-40472b29-28d9-465f-a704-5deae14f0daa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87b7c3ec77d4493dac92bb61d0b376be'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:06.377966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-40472b29-28d9-465f-a704-5deae14f0daa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87b7c3ec77d4493dac92bb61d0b376be'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:06.394187 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1116/2261] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:32:06 2022] DELETE /identity/v3/users/87b7c3ec77d4493dac92bb61d0b376be => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:06.566621 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d4387-cc7b-4d29-aad2-5da06455cbdd service nova] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:06.583929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d4387-cc7b-4d29-aad2-5da06455cbdd service nova] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:06.655930 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374730f-9fe7-41dd-a591-63e10de55e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:06.656812 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374730f-9fe7-41dd-a591-63e10de55e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:06.657122 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374730f-9fe7-41dd-a591-63e10de55e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:06.657395 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374730f-9fe7-41dd-a591-63e10de55e65 admin admin] PATH_INFO: `/v3/projects/d28d5139952d450fa3b7da38a62d7a03` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:06.658571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374730f-9fe7-41dd-a591-63e10de55e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d28d5139952d450fa3b7da38a62d7a03)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:06.660505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374730f-9fe7-41dd-a591-63e10de55e65 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:06.705983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c374730f-9fe7-41dd-a591-63e10de55e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd28d5139952d450fa3b7da38a62d7a03', 'request_id': 'req-c374730f-9fe7-41dd-a591-63e10de55e65'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:06.711156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c374730f-9fe7-41dd-a591-63e10de55e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd28d5139952d450fa3b7da38a62d7a03', 'request_id': 'req-c374730f-9fe7-41dd-a591-63e10de55e65'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:06.719373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c374730f-9fe7-41dd-a591-63e10de55e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd28d5139952d450fa3b7da38a62d7a03', 'request_id': 'req-c374730f-9fe7-41dd-a591-63e10de55e65'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:06.731137 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c374730f-9fe7-41dd-a591-63e10de55e65 admin admin] The token cache is being invalidate because project d28d5139952d450fa3b7da38a62d7a03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:32:06.731419 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c374730f-9fe7-41dd-a591-63e10de55e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:06.732206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c374730f-9fe7-41dd-a591-63e10de55e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:06.733760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1146/2262] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:32:06 2022] DELETE /identity/v3/projects/d28d5139952d450fa3b7da38a62d7a03 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:06.738838 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bf27d-d53e-4b92-a87b-bcb3635618b1 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:06.739139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bf27d-d53e-4b92-a87b-bcb3635618b1 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:06.739435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bf27d-d53e-4b92-a87b-bcb3635618b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:06.778210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-949bf27d-d53e-4b92-a87b-bcb3635618b1 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:06.779386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-949bf27d-d53e-4b92-a87b-bcb3635618b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:06.839022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1117/2263] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:32:06.846690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590374d6-d565-46a9-a88d-135ee5ae4633 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:06.847053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590374d6-d565-46a9-a88d-135ee5ae4633 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:06.847401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590374d6-d565-46a9-a88d-135ee5ae4633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:06.865532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-590374d6-d565-46a9-a88d-135ee5ae4633 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:06.866713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-590374d6-d565-46a9-a88d-135ee5ae4633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:06.876128 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1147/2264] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:06.880500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bf27d-d53e-4b92-a87b-bcb3635618b1 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:06.890851 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bf27d-d53e-4b92-a87b-bcb3635618b1 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:06.896326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69ee2e-c434-492b-91fd-1ada4ae2643d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:06.897243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69ee2e-c434-492b-91fd-1ada4ae2643d admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:06.897534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69ee2e-c434-492b-91fd-1ada4ae2643d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:06.897811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69ee2e-c434-492b-91fd-1ada4ae2643d admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:06.898539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69ee2e-c434-492b-91fd-1ada4ae2643d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:32:06.898847 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69ee2e-c434-492b-91fd-1ada4ae2643d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:06.900723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69ee2e-c434-492b-91fd-1ada4ae2643d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:06.912434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1118/2265] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:32:06.916878 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590374d6-d565-46a9-a88d-135ee5ae4633 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:06.926111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590374d6-d565-46a9-a88d-135ee5ae4633 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:06.930811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22df1a6-db5c-44ce-ae87-c54ced4d6fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:06.932242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22df1a6-db5c-44ce-ae87-c54ced4d6fcc admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:06.932618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22df1a6-db5c-44ce-ae87-c54ced4d6fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:06.932840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22df1a6-db5c-44ce-ae87-c54ced4d6fcc admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:06.933551 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22df1a6-db5c-44ce-ae87-c54ced4d6fcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:06.935483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22df1a6-db5c-44ce-ae87-c54ced4d6fcc admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:06.965564 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1148/2266] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:32:06 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:06.966735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69ee2e-c434-492b-91fd-1ada4ae2643d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:06.976344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69ee2e-c434-492b-91fd-1ada4ae2643d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:07.031764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f439ee-d7e5-4763-9cab-cfa02b85e22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:07.032477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f439ee-d7e5-4763-9cab-cfa02b85e22f admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:07.032763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f439ee-d7e5-4763-9cab-cfa02b85e22f admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:07.033040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f439ee-d7e5-4763-9cab-cfa02b85e22f admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:07.033729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f439ee-d7e5-4763-9cab-cfa02b85e22f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:07.035624 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f439ee-d7e5-4763-9cab-cfa02b85e22f admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:07.057983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1119/2267] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:32:06 2022] POST /identity/v3/users => generated 303 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:07.062432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22df1a6-db5c-44ce-ae87-c54ced4d6fcc admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:07.077242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22df1a6-db5c-44ce-ae87-c54ced4d6fcc admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:07.082108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddaddd9-a3be-49cd-829a-0a6a0db93282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:07.082962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddaddd9-a3be-49cd-829a-0a6a0db93282 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:07.083247 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddaddd9-a3be-49cd-829a-0a6a0db93282 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:07.083521 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddaddd9-a3be-49cd-829a-0a6a0db93282 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:07.084282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddaddd9-a3be-49cd-829a-0a6a0db93282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:07.085868 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddaddd9-a3be-49cd-829a-0a6a0db93282 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:07.097859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1149/2268] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:07.102071 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f439ee-d7e5-4763-9cab-cfa02b85e22f admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:07.114301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f439ee-d7e5-4763-9cab-cfa02b85e22f admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:07.119104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b085b38-b894-4463-b309-a728c2e90e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:07.119773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b085b38-b894-4463-b309-a728c2e90e04 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:07.120113 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b085b38-b894-4463-b309-a728c2e90e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:07.120395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b085b38-b894-4463-b309-a728c2e90e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:07.121238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b085b38-b894-4463-b309-a728c2e90e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:07.122832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b085b38-b894-4463-b309-a728c2e90e04 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:07.132836 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1120/2269] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:32:07.137790 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddaddd9-a3be-49cd-829a-0a6a0db93282 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:07.147464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddaddd9-a3be-49cd-829a-0a6a0db93282 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:07.152732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848b8e1-dc0d-4bc4-afcf-c785b67e8728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:07.153611 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848b8e1-dc0d-4bc4-afcf-c785b67e8728 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:07.153891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848b8e1-dc0d-4bc4-afcf-c785b67e8728 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:07.154166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848b8e1-dc0d-4bc4-afcf-c785b67e8728 admin admin] PATH_INFO: `/v3/projects/97fd404f1d274a97a2d5bc1168eb3e63/users/41b8f773877f432b986eaec624bbbf10/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:07.168047 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848b8e1-dc0d-4bc4-afcf-c785b67e8728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97fd404f1d274a97a2d5bc1168eb3e63, user_id=41b8f773877f432b986eaec624bbbf10, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:07.170695 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848b8e1-dc0d-4bc4-afcf-c785b67e8728 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:07.184126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1150/2270] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:07 2022] PUT /identity/v3/projects/97fd404f1d274a97a2d5bc1168eb3e63/users/41b8f773877f432b986eaec624bbbf10/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:07.191657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a30d7-7c03-4adb-b854-d8ae99387c87 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:07.191978 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a30d7-7c03-4adb-b854-d8ae99387c87 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:07.192256 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a30d7-7c03-4adb-b854-d8ae99387c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:07.210173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-193a30d7-7c03-4adb-b854-d8ae99387c87 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:07.211363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-193a30d7-7c03-4adb-b854-d8ae99387c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:07.278919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1121/2271] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:07.285762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848b8e1-dc0d-4bc4-afcf-c785b67e8728 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:07.287653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e848b8e1-dc0d-4bc4-afcf-c785b67e8728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:07.396891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848b8e1-dc0d-4bc4-afcf-c785b67e8728 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:07.404142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bc1cd-81f2-4960-95c6-b5063c0fb7d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:07.405471 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bc1cd-81f2-4960-95c6-b5063c0fb7d7 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:07.405764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bc1cd-81f2-4960-95c6-b5063c0fb7d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:07.406043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bc1cd-81f2-4960-95c6-b5063c0fb7d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:07.418063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8bc1cd-81f2-4960-95c6-b5063c0fb7d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:07.419494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8bc1cd-81f2-4960-95c6-b5063c0fb7d7 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:07.432853 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1151/2272] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:32:08.385909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de69b5d-d4e2-4176-ba3d-d40bea1cce5a None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:08.386885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de69b5d-d4e2-4176-ba3d-d40bea1cce5a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:08.387328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de69b5d-d4e2-4176-ba3d-d40bea1cce5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:08.418175 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-1de69b5d-d4e2-4176-ba3d-d40bea1cce5a None None] MFA Rules not processed for user `41b8f773877f432b986eaec624bbbf10`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:08.419430 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1de69b5d-d4e2-4176-ba3d-d40bea1cce5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:08.496958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1122/2273] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:08 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:08.498521 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bc1cd-81f2-4960-95c6-b5063c0fb7d7 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:08.508662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bc1cd-81f2-4960-95c6-b5063c0fb7d7 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:08.513625 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d942-2a62-4c95-8ca0-590ca2c41be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:08.514314 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d942-2a62-4c95-8ca0-590ca2c41be5 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:08.514590 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d942-2a62-4c95-8ca0-590ca2c41be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:08.514860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0d942-2a62-4c95-8ca0-590ca2c41be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:08.515554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0d942-2a62-4c95-8ca0-590ca2c41be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:08.517609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0d942-2a62-4c95-8ca0-590ca2c41be5 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:08.548796 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1152/2274] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:32:08 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:08.549663 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de69b5d-d4e2-4176-ba3d-d40bea1cce5a None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:08.564796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de69b5d-d4e2-4176-ba3d-d40bea1cce5a None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:08.624329 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e8280-195b-44d5-b5df-b90aaf5fa41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:08.625001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e8280-195b-44d5-b5df-b90aaf5fa41b admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:08.625280 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e8280-195b-44d5-b5df-b90aaf5fa41b admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:08.625613 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e8280-195b-44d5-b5df-b90aaf5fa41b admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:08.626298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e8280-195b-44d5-b5df-b90aaf5fa41b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:08.628231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e8280-195b-44d5-b5df-b90aaf5fa41b admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:08.648809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1123/2275] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:32:08 2022] POST /identity/v3/users => generated 309 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:32:08.653448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d942-2a62-4c95-8ca0-590ca2c41be5 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:08.663380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0d942-2a62-4c95-8ca0-590ca2c41be5 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:08.667852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5d16d-2091-49c4-ba72-d6daa4cdd2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:08.668541 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5d16d-2091-49c4-ba72-d6daa4cdd2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:08.668825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5d16d-2091-49c4-ba72-d6daa4cdd2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:08.669101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5d16d-2091-49c4-ba72-d6daa4cdd2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:08.669822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5d16d-2091-49c4-ba72-d6daa4cdd2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:08.671404 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5d16d-2091-49c4-ba72-d6daa4cdd2e4 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:08.682953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1153/2276] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:08.687182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e8280-195b-44d5-b5df-b90aaf5fa41b admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:08.698237 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e8280-195b-44d5-b5df-b90aaf5fa41b admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:08.703032 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be563927-5d40-4065-8c89-4f1180c1d4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:08.704018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be563927-5d40-4065-8c89-4f1180c1d4b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:08.704364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be563927-5d40-4065-8c89-4f1180c1d4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:08.704760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be563927-5d40-4065-8c89-4f1180c1d4b1 admin admin] PATH_INFO: `/v3/projects/e759b417484e4d728f862f8f6fd4e239/users/cae33f1faaed4a9893abc38333df6fea/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:08.720813 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be563927-5d40-4065-8c89-4f1180c1d4b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e759b417484e4d728f862f8f6fd4e239, user_id=cae33f1faaed4a9893abc38333df6fea, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:08.723469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be563927-5d40-4065-8c89-4f1180c1d4b1 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:08.738449 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1124/2277] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:08 2022] PUT /identity/v3/projects/e759b417484e4d728f862f8f6fd4e239/users/cae33f1faaed4a9893abc38333df6fea/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:32:08.742471 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5d16d-2091-49c4-ba72-d6daa4cdd2e4 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:08.756081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5d16d-2091-49c4-ba72-d6daa4cdd2e4 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:08.816006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b774264-5850-4f3c-aee1-3d554bb63e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:08.816815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b774264-5850-4f3c-aee1-3d554bb63e48 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:08.817122 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b774264-5850-4f3c-aee1-3d554bb63e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:08.817369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b774264-5850-4f3c-aee1-3d554bb63e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:08.818092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b774264-5850-4f3c-aee1-3d554bb63e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:08.819689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b774264-5850-4f3c-aee1-3d554bb63e48 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:08.830161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1154/2278] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:08.834461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be563927-5d40-4065-8c89-4f1180c1d4b1 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:08.845315 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be563927-5d40-4065-8c89-4f1180c1d4b1 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:08.850335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a039f-b576-4d88-9bc7-0ad1be77f59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:08.851249 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a039f-b576-4d88-9bc7-0ad1be77f59e admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:08.851532 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a039f-b576-4d88-9bc7-0ad1be77f59e admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:08.851809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a039f-b576-4d88-9bc7-0ad1be77f59e admin admin] PATH_INFO: `/v3/projects/e759b417484e4d728f862f8f6fd4e239/users/cae33f1faaed4a9893abc38333df6fea/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:08.854485 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a039f-b576-4d88-9bc7-0ad1be77f59e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e759b417484e4d728f862f8f6fd4e239, user_id=cae33f1faaed4a9893abc38333df6fea, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:08.857471 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a039f-b576-4d88-9bc7-0ad1be77f59e admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:08.868724 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1125/2279] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:08 2022] PUT /identity/v3/projects/e759b417484e4d728f862f8f6fd4e239/users/cae33f1faaed4a9893abc38333df6fea/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:32:09.623423 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3d1ca-7e91-4b68-b7d0-b862fde89bd0 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:09.624951 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3d1ca-7e91-4b68-b7d0-b862fde89bd0 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:09.625487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3d1ca-7e91-4b68-b7d0-b862fde89bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:09.667544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2ec3d1ca-7e91-4b68-b7d0-b862fde89bd0 None None] MFA Rules not processed for user `cae33f1faaed4a9893abc38333df6fea`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:09.669331 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2ec3d1ca-7e91-4b68-b7d0-b862fde89bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:09.783988 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1155/2280] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:09 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:10.841559 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a039f-b576-4d88-9bc7-0ad1be77f59e admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:10.908355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a039f-b576-4d88-9bc7-0ad1be77f59e admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:10.913151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:10.914021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:10.914300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:10.914570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] PATH_INFO: `/v3/users/41b8f773877f432b986eaec624bbbf10` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:10.915775 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41b8f773877f432b986eaec624bbbf10)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:10.917914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:10.963888 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41b8f773877f432b986eaec624bbbf10'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:10.964591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41b8f773877f432b986eaec624bbbf10'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:10.965177 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41b8f773877f432b986eaec624bbbf10'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:10.973100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41b8f773877f432b986eaec624bbbf10'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:10.980068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41b8f773877f432b986eaec624bbbf10'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:10.996657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1126/2281] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:32:10 2022] DELETE /identity/v3/users/41b8f773877f432b986eaec624bbbf10 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:11.168369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3d1ca-7e91-4b68-b7d0-b862fde89bd0 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:11.179217 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3d1ca-7e91-4b68-b7d0-b862fde89bd0 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:11.235257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:11.236122 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:11.236512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:11.236794 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] PATH_INFO: `/v3/projects/97fd404f1d274a97a2d5bc1168eb3e63` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:11.237954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97fd404f1d274a97a2d5bc1168eb3e63)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:11.239922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:11.283693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97fd404f1d274a97a2d5bc1168eb3e63', 'request_id': 'req-f5a17955-18c1-47c8-8fa5-b127e15bf674'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.289340 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97fd404f1d274a97a2d5bc1168eb3e63', 'request_id': 'req-f5a17955-18c1-47c8-8fa5-b127e15bf674'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.297362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97fd404f1d274a97a2d5bc1168eb3e63', 'request_id': 'req-f5a17955-18c1-47c8-8fa5-b127e15bf674'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.310244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] The token cache is being invalidate because project 97fd404f1d274a97a2d5bc1168eb3e63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:32:11.310605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.311287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.313360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1156/2282] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:32:11 2022] DELETE /identity/v3/projects/97fd404f1d274a97a2d5bc1168eb3e63 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:11.317632 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:11.320091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:11.335387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c31d50-64ab-421b-af6a-bbf8558387f3 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:11.394120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f291c-4945-453f-93b3-fe067c0e67af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:11.395000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f291c-4945-453f-93b3-fe067c0e67af admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:11.395280 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f291c-4945-453f-93b3-fe067c0e67af admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:11.395553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f291c-4945-453f-93b3-fe067c0e67af admin admin] PATH_INFO: `/v3/users/cae33f1faaed4a9893abc38333df6fea` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:11.396767 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f291c-4945-453f-93b3-fe067c0e67af admin admin] RBAC: Authorizing `identity:delete_user(user_id=cae33f1faaed4a9893abc38333df6fea)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:11.398702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f291c-4945-453f-93b3-fe067c0e67af admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:11.444077 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-403f291c-4945-453f-93b3-fe067c0e67af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cae33f1faaed4a9893abc38333df6fea'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.445028 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-403f291c-4945-453f-93b3-fe067c0e67af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cae33f1faaed4a9893abc38333df6fea'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.446019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-403f291c-4945-453f-93b3-fe067c0e67af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cae33f1faaed4a9893abc38333df6fea'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.454417 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-403f291c-4945-453f-93b3-fe067c0e67af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cae33f1faaed4a9893abc38333df6fea'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.461303 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-403f291c-4945-453f-93b3-fe067c0e67af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cae33f1faaed4a9893abc38333df6fea'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.478991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1127/2283] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:32:11 2022] DELETE /identity/v3/users/cae33f1faaed4a9893abc38333df6fea => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:11.656214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:11.656214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:11.678682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a17955-18c1-47c8-8fa5-b127e15bf674 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:11.741235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fdf88-ae79-4e6e-b791-af83d67e521a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:11.742108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fdf88-ae79-4e6e-b791-af83d67e521a admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:11.742393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fdf88-ae79-4e6e-b791-af83d67e521a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:11.742676 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fdf88-ae79-4e6e-b791-af83d67e521a admin admin] PATH_INFO: `/v3/projects/e759b417484e4d728f862f8f6fd4e239` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:11.744703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fdf88-ae79-4e6e-b791-af83d67e521a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e759b417484e4d728f862f8f6fd4e239)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:11.746706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fdf88-ae79-4e6e-b791-af83d67e521a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:11.799124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b22fdf88-ae79-4e6e-b791-af83d67e521a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e759b417484e4d728f862f8f6fd4e239', 'request_id': 'req-b22fdf88-ae79-4e6e-b791-af83d67e521a'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.807510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b22fdf88-ae79-4e6e-b791-af83d67e521a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e759b417484e4d728f862f8f6fd4e239', 'request_id': 'req-b22fdf88-ae79-4e6e-b791-af83d67e521a'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.818116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b22fdf88-ae79-4e6e-b791-af83d67e521a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e759b417484e4d728f862f8f6fd4e239', 'request_id': 'req-b22fdf88-ae79-4e6e-b791-af83d67e521a'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.830684 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b22fdf88-ae79-4e6e-b791-af83d67e521a admin admin] The token cache is being invalidate because project e759b417484e4d728f862f8f6fd4e239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:32:11.830925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b22fdf88-ae79-4e6e-b791-af83d67e521a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.831549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-b22fdf88-ae79-4e6e-b791-af83d67e521a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:11.833142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1157/2284] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:32:11 2022] DELETE /identity/v3/projects/e759b417484e4d728f862f8f6fd4e239 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:11.838859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629ea96-0449-41c0-8492-2bec81711613 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:11.839152 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629ea96-0449-41c0-8492-2bec81711613 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:11.839457 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629ea96-0449-41c0-8492-2bec81711613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:11.858680 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-6629ea96-0449-41c0-8492-2bec81711613 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:11.859959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-6629ea96-0449-41c0-8492-2bec81711613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:11.928095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1128/2285] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:32:11.936326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b20447-d60c-45d7-94af-229e8597de97 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:11.936615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b20447-d60c-45d7-94af-229e8597de97 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:11.936895 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b20447-d60c-45d7-94af-229e8597de97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:11.954918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e1b20447-d60c-45d7-94af-229e8597de97 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:11.956142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e1b20447-d60c-45d7-94af-229e8597de97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:11.964926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1158/2286] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:11.969062 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629ea96-0449-41c0-8492-2bec81711613 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:11.979185 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629ea96-0449-41c0-8492-2bec81711613 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:11.983671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac683f-8002-434f-b48a-d131fbf90ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:11.984357 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac683f-8002-434f-b48a-d131fbf90ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:11.984640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac683f-8002-434f-b48a-d131fbf90ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:11.984917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac683f-8002-434f-b48a-d131fbf90ba2 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:11.985632 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac683f-8002-434f-b48a-d131fbf90ba2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:32:11.985938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac683f-8002-434f-b48a-d131fbf90ba2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:11.987514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac683f-8002-434f-b48a-d131fbf90ba2 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:11.998942 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1129/2287] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:32:12.002834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b20447-d60c-45d7-94af-229e8597de97 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:12.011857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b20447-d60c-45d7-94af-229e8597de97 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:12.016452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f072c-fdc9-4f28-8dde-ad4f6f881047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:12.017092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f072c-fdc9-4f28-8dde-ad4f6f881047 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.017378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f072c-fdc9-4f28-8dde-ad4f6f881047 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.017652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f072c-fdc9-4f28-8dde-ad4f6f881047 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.018335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f072c-fdc9-4f28-8dde-ad4f6f881047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:12.020239 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f072c-fdc9-4f28-8dde-ad4f6f881047 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:12.045615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1159/2288] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:32:12 2022] POST /identity/v3/projects => generated 381 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:12.049609 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac683f-8002-434f-b48a-d131fbf90ba2 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:12.059515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac683f-8002-434f-b48a-d131fbf90ba2 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:12.115243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89308a6-fb94-4ea5-935a-102deed17033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:12.115938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89308a6-fb94-4ea5-935a-102deed17033 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.116333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89308a6-fb94-4ea5-935a-102deed17033 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.116732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89308a6-fb94-4ea5-935a-102deed17033 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.117577 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89308a6-fb94-4ea5-935a-102deed17033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:12.120515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89308a6-fb94-4ea5-935a-102deed17033 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:12.145344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1130/2289] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:32:12 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:12.146585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f072c-fdc9-4f28-8dde-ad4f6f881047 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:12.157416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f072c-fdc9-4f28-8dde-ad4f6f881047 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:12.161914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e143b-0b75-4582-af55-c16560c1bcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:12.162573 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e143b-0b75-4582-af55-c16560c1bcd3 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.162854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e143b-0b75-4582-af55-c16560c1bcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.163129 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e143b-0b75-4582-af55-c16560c1bcd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.163854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e143b-0b75-4582-af55-c16560c1bcd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:12.166056 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e143b-0b75-4582-af55-c16560c1bcd3 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:12.178683 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1160/2290] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:12.179797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89308a6-fb94-4ea5-935a-102deed17033 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:12.189882 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89308a6-fb94-4ea5-935a-102deed17033 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:12.194476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc98b9f-2981-4086-b3a0-011a6290c3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:12.195124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc98b9f-2981-4086-b3a0-011a6290c3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.195405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc98b9f-2981-4086-b3a0-011a6290c3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.195676 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc98b9f-2981-4086-b3a0-011a6290c3b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.196427 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc98b9f-2981-4086-b3a0-011a6290c3b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:12.198548 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc98b9f-2981-4086-b3a0-011a6290c3b9 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:12.208398 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1131/2291] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:32:12.213099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e143b-0b75-4582-af55-c16560c1bcd3 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:12.222993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e143b-0b75-4582-af55-c16560c1bcd3 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:12.227567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa81e8-c4ec-418a-9aa1-30b066d6802e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:12.229046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa81e8-c4ec-418a-9aa1-30b066d6802e admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.229324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa81e8-c4ec-418a-9aa1-30b066d6802e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.229595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa81e8-c4ec-418a-9aa1-30b066d6802e admin admin] PATH_INFO: `/v3/projects/afee4df811e04ef9bbf4fa86ae322f71/users/5efa52f44e2646d4b319bf67ab0d2871/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.243219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa81e8-c4ec-418a-9aa1-30b066d6802e admin admin] RBAC: Authorizing `identity:create_grant(project_id=afee4df811e04ef9bbf4fa86ae322f71, user_id=5efa52f44e2646d4b319bf67ab0d2871, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:12.245908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa81e8-c4ec-418a-9aa1-30b066d6802e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:12.258269 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1161/2292] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:12 2022] PUT /identity/v3/projects/afee4df811e04ef9bbf4fa86ae322f71/users/5efa52f44e2646d4b319bf67ab0d2871/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:12.268803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbf198-7099-4d26-b455-941452e05dde None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.269094 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbf198-7099-4d26-b455-941452e05dde None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.269395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbf198-7099-4d26-b455-941452e05dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.296604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-3ffbf198-7099-4d26-b455-941452e05dde None None] MFA Rules not processed for user `5efa52f44e2646d4b319bf67ab0d2871`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:12.298242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3ffbf198-7099-4d26-b455-941452e05dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:12.359714 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1132/2293] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:12.364151 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa81e8-c4ec-418a-9aa1-30b066d6802e admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:12.421811 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa81e8-c4ec-418a-9aa1-30b066d6802e admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:12.426320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221591ab-2b58-4bb8-b8e5-ed08069d36d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:12.426968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221591ab-2b58-4bb8-b8e5-ed08069d36d2 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.427249 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221591ab-2b58-4bb8-b8e5-ed08069d36d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.427523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221591ab-2b58-4bb8-b8e5-ed08069d36d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.428238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221591ab-2b58-4bb8-b8e5-ed08069d36d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:12.430115 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221591ab-2b58-4bb8-b8e5-ed08069d36d2 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:12.454088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1162/2294] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:32:12 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:12.458157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbf198-7099-4d26-b455-941452e05dde None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:12.468615 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbf198-7099-4d26-b455-941452e05dde None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:12.522708 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53a8d4-b067-451f-b759-fc2d350a421d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:12.523413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53a8d4-b067-451f-b759-fc2d350a421d admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.523702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53a8d4-b067-451f-b759-fc2d350a421d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.524046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53a8d4-b067-451f-b759-fc2d350a421d admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.524749 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53a8d4-b067-451f-b759-fc2d350a421d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:12.527359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b53a8d4-b067-451f-b759-fc2d350a421d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:12.548173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1133/2295] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:32:12 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:12.552292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221591ab-2b58-4bb8-b8e5-ed08069d36d2 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:12.562490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221591ab-2b58-4bb8-b8e5-ed08069d36d2 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:12.567066 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b1a7c-f2fe-4570-aa80-a61badef4a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:12.567729 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b1a7c-f2fe-4570-aa80-a61badef4a02 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.568049 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b1a7c-f2fe-4570-aa80-a61badef4a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.568326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b1a7c-f2fe-4570-aa80-a61badef4a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.569071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b1a7c-f2fe-4570-aa80-a61badef4a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:12.570639 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b1a7c-f2fe-4570-aa80-a61badef4a02 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:12.583160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1163/2296] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:12.587190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53a8d4-b067-451f-b759-fc2d350a421d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:12.598466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53a8d4-b067-451f-b759-fc2d350a421d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:12.603711 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77df30-bac5-4ecb-9aaf-4d1114f785b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:12.604685 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77df30-bac5-4ecb-9aaf-4d1114f785b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.605193 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77df30-bac5-4ecb-9aaf-4d1114f785b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.605481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77df30-bac5-4ecb-9aaf-4d1114f785b1 admin admin] PATH_INFO: `/v3/projects/e19e0cdd8e5f4908a00c7807c3635961/users/4cafdd61575544d4a427ee8e7821bc21/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.620537 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77df30-bac5-4ecb-9aaf-4d1114f785b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19e0cdd8e5f4908a00c7807c3635961, user_id=4cafdd61575544d4a427ee8e7821bc21, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:12.623203 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77df30-bac5-4ecb-9aaf-4d1114f785b1 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:12.638399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1134/2297] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:12 2022] PUT /identity/v3/projects/e19e0cdd8e5f4908a00c7807c3635961/users/4cafdd61575544d4a427ee8e7821bc21/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:12.642538 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b1a7c-f2fe-4570-aa80-a61badef4a02 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:12.653051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b1a7c-f2fe-4570-aa80-a61badef4a02 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:12.718802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25e058-4289-46c1-ad63-ddc7e0891fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:12.719483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25e058-4289-46c1-ad63-ddc7e0891fad admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.719767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25e058-4289-46c1-ad63-ddc7e0891fad admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.720085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25e058-4289-46c1-ad63-ddc7e0891fad admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.720816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25e058-4289-46c1-ad63-ddc7e0891fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:12.722491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25e058-4289-46c1-ad63-ddc7e0891fad admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:12.733033 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1164/2298] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:12.737250 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77df30-bac5-4ecb-9aaf-4d1114f785b1 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:12.749659 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77df30-bac5-4ecb-9aaf-4d1114f785b1 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:12.754899 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c413a6-9c9b-47fc-a96b-b67fcea97dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:12.755791 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c413a6-9c9b-47fc-a96b-b67fcea97dfa admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.756102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c413a6-9c9b-47fc-a96b-b67fcea97dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.756377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c413a6-9c9b-47fc-a96b-b67fcea97dfa admin admin] PATH_INFO: `/v3/projects/e19e0cdd8e5f4908a00c7807c3635961/users/4cafdd61575544d4a427ee8e7821bc21/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.758368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c413a6-9c9b-47fc-a96b-b67fcea97dfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19e0cdd8e5f4908a00c7807c3635961, user_id=4cafdd61575544d4a427ee8e7821bc21, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:12.761305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c413a6-9c9b-47fc-a96b-b67fcea97dfa admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:12.771921 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1135/2299] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:12 2022] PUT /identity/v3/projects/e19e0cdd8e5f4908a00c7807c3635961/users/4cafdd61575544d4a427ee8e7821bc21/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:12.783734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b2640-e923-4660-9998-6f6406e7599b None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.784071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b2640-e923-4660-9998-6f6406e7599b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.784352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b2640-e923-4660-9998-6f6406e7599b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.811674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-1a4b2640-e923-4660-9998-6f6406e7599b None None] MFA Rules not processed for user `4cafdd61575544d4a427ee8e7821bc21`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:12.812898 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1a4b2640-e923-4660-9998-6f6406e7599b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:12.882415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1165/2300] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:12.895054 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c413a6-9c9b-47fc-a96b-b67fcea97dfa admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:12.897702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-98c413a6-9c9b-47fc-a96b-b67fcea97dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:12.977570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c413a6-9c9b-47fc-a96b-b67fcea97dfa admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:12.982229 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373575d2-08de-44bb-a998-709dab8d356e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:12.982874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373575d2-08de-44bb-a998-709dab8d356e service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:12.983159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373575d2-08de-44bb-a998-709dab8d356e service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:12.983432 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373575d2-08de-44bb-a998-709dab8d356e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:12.992385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373575d2-08de-44bb-a998-709dab8d356e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:12.994135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373575d2-08de-44bb-a998-709dab8d356e service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:13.003490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1136/2301] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:32:13.077206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b2640-e923-4660-9998-6f6406e7599b None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:13.088233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b2640-e923-4660-9998-6f6406e7599b None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:13.092253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e71d19-4f56-4ee7-a296-8503f759e75d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:13.092902 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e71d19-4f56-4ee7-a296-8503f759e75d service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:13.093188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e71d19-4f56-4ee7-a296-8503f759e75d service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:13.093567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e71d19-4f56-4ee7-a296-8503f759e75d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:13.154411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e71d19-4f56-4ee7-a296-8503f759e75d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:13.155852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e71d19-4f56-4ee7-a296-8503f759e75d service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:13.165629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1166/2302] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:32:16.021121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dffcfc-2390-497b-9220-326e9c81cadc None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:16.021615 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dffcfc-2390-497b-9220-326e9c81cadc None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:16.021715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dffcfc-2390-497b-9220-326e9c81cadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:16.039729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-43dffcfc-2390-497b-9220-326e9c81cadc None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:16.041792 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-43dffcfc-2390-497b-9220-326e9c81cadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:16.099100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1137/2303] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:32:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:16.112418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e71d19-4f56-4ee7-a296-8503f759e75d service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:16.113870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-65e71d19-4f56-4ee7-a296-8503f759e75d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:16.194543 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e71d19-4f56-4ee7-a296-8503f759e75d service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:16.199978 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99513e13-40a6-452f-8940-8f51bc8b930c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:16.200637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99513e13-40a6-452f-8940-8f51bc8b930c service swift] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:16.200919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99513e13-40a6-452f-8940-8f51bc8b930c service swift] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:16.201194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99513e13-40a6-452f-8940-8f51bc8b930c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:16.209607 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99513e13-40a6-452f-8940-8f51bc8b930c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:16.211183 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99513e13-40a6-452f-8940-8f51bc8b930c service swift] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:16.219647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1167/2304] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:32:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:32:23.076947 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e95a92-d53a-4827-b87b-8de2d92c76c8 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:23.077522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e95a92-d53a-4827-b87b-8de2d92c76c8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:23.077729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e95a92-d53a-4827-b87b-8de2d92c76c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:23.097617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-37e95a92-d53a-4827-b87b-8de2d92c76c8 None None] MFA Rules not processed for user `c56a403eeee84720ba765e2d4367df74`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:23.099088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-37e95a92-d53a-4827-b87b-8de2d92c76c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:23.185347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1138/2305] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:23.362450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99513e13-40a6-452f-8940-8f51bc8b930c service swift] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:23.447812 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99513e13-40a6-452f-8940-8f51bc8b930c service swift] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:23.453507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:23.454395 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:23.455322 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:23.455612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] PATH_INFO: `/v3/users/5efa52f44e2646d4b319bf67ab0d2871` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:23.457000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5efa52f44e2646d4b319bf67ab0d2871)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:23.459154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:23.512487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5efa52f44e2646d4b319bf67ab0d2871'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:23.513170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5efa52f44e2646d4b319bf67ab0d2871'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:23.523397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5efa52f44e2646d4b319bf67ab0d2871'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:23.532393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5efa52f44e2646d4b319bf67ab0d2871'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:23.532798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5efa52f44e2646d4b319bf67ab0d2871'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:23.555215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1168/2306] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:32:23 2022] DELETE /identity/v3/users/5efa52f44e2646d4b319bf67ab0d2871 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:23.557641 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d4367-65f9-42c3-be5d-864aaf38e2b5 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:23.558173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d4367-65f9-42c3-be5d-864aaf38e2b5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:23.558772 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d4367-65f9-42c3-be5d-864aaf38e2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:23.579754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-eb8d4367-65f9-42c3-be5d-864aaf38e2b5 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:23.580985 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-eb8d4367-65f9-42c3-be5d-864aaf38e2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:23.657228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1139/2307] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:23.671139 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:23.672899 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:23.773644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddb90e-4436-4c52-81bf-611d623cc9c1 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:23.790551 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b881b76-652c-46ca-ad5c-99052c44d1f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:23.794787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b881b76-652c-46ca-ad5c-99052c44d1f5 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:23.794787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b881b76-652c-46ca-ad5c-99052c44d1f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:23.794787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b881b76-652c-46ca-ad5c-99052c44d1f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:23.833231 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b881b76-652c-46ca-ad5c-99052c44d1f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:23.834868 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b881b76-652c-46ca-ad5c-99052c44d1f5 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:23.855346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1169/2308] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:32:24.398390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d4367-65f9-42c3-be5d-864aaf38e2b5 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:24.402172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-eb8d4367-65f9-42c3-be5d-864aaf38e2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:24.428085 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d4367-65f9-42c3-be5d-864aaf38e2b5 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:24.450745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:24.451641 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:24.452002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:24.452637 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] PATH_INFO: `/v3/projects/afee4df811e04ef9bbf4fa86ae322f71` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:24.454076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] RBAC: Authorizing `identity:delete_project(project_id=afee4df811e04ef9bbf4fa86ae322f71)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:24.458528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:24.532814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afee4df811e04ef9bbf4fa86ae322f71', 'request_id': 'req-ae822d6b-58b6-4366-9d71-e7f08a50d90d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:24.539115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afee4df811e04ef9bbf4fa86ae322f71', 'request_id': 'req-ae822d6b-58b6-4366-9d71-e7f08a50d90d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:24.546623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afee4df811e04ef9bbf4fa86ae322f71', 'request_id': 'req-ae822d6b-58b6-4366-9d71-e7f08a50d90d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:24.567552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] The token cache is being invalidate because project afee4df811e04ef9bbf4fa86ae322f71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:32:24.568178 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:24.569473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:24.571936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1140/2309] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:32:24 2022] DELETE /identity/v3/projects/afee4df811e04ef9bbf4fa86ae322f71 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:24.576649 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b881b76-652c-46ca-ad5c-99052c44d1f5 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:24.589334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b881b76-652c-46ca-ad5c-99052c44d1f5 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:24.662441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d77d1-17d7-4cbb-86c4-b9ac4cad3d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:24.663328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d77d1-17d7-4cbb-86c4-b9ac4cad3d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:24.663608 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d77d1-17d7-4cbb-86c4-b9ac4cad3d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:24.663879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d77d1-17d7-4cbb-86c4-b9ac4cad3d82 admin admin] PATH_INFO: `/v3/users/4cafdd61575544d4a427ee8e7821bc21` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:24.665110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d77d1-17d7-4cbb-86c4-b9ac4cad3d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cafdd61575544d4a427ee8e7821bc21)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:24.667211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d77d1-17d7-4cbb-86c4-b9ac4cad3d82 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:24.733200 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4e2d77d1-17d7-4cbb-86c4-b9ac4cad3d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cafdd61575544d4a427ee8e7821bc21'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:24.733889 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4e2d77d1-17d7-4cbb-86c4-b9ac4cad3d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cafdd61575544d4a427ee8e7821bc21'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:24.749062 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4e2d77d1-17d7-4cbb-86c4-b9ac4cad3d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cafdd61575544d4a427ee8e7821bc21'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:24.763178 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4e2d77d1-17d7-4cbb-86c4-b9ac4cad3d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cafdd61575544d4a427ee8e7821bc21'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:24.763950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4e2d77d1-17d7-4cbb-86c4-b9ac4cad3d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cafdd61575544d4a427ee8e7821bc21'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:24.800091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1170/2310] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:32:24 2022] DELETE /identity/v3/users/4cafdd61575544d4a427ee8e7821bc21 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:25.154336 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:25.156406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:25.198155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae822d6b-58b6-4366-9d71-e7f08a50d90d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:25.361637 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d24ede-5002-4f02-99c0-9d9eb39b09fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:25.362211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d24ede-5002-4f02-99c0-9d9eb39b09fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:25.362536 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d24ede-5002-4f02-99c0-9d9eb39b09fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:25.362819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d24ede-5002-4f02-99c0-9d9eb39b09fc admin admin] PATH_INFO: `/v3/projects/e19e0cdd8e5f4908a00c7807c3635961` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:25.364129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d24ede-5002-4f02-99c0-9d9eb39b09fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19e0cdd8e5f4908a00c7807c3635961)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:25.366161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d24ede-5002-4f02-99c0-9d9eb39b09fc admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:25.447774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-22d24ede-5002-4f02-99c0-9d9eb39b09fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19e0cdd8e5f4908a00c7807c3635961', 'request_id': 'req-22d24ede-5002-4f02-99c0-9d9eb39b09fc'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:25.456815 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-22d24ede-5002-4f02-99c0-9d9eb39b09fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19e0cdd8e5f4908a00c7807c3635961', 'request_id': 'req-22d24ede-5002-4f02-99c0-9d9eb39b09fc'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:25.465420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-22d24ede-5002-4f02-99c0-9d9eb39b09fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19e0cdd8e5f4908a00c7807c3635961', 'request_id': 'req-22d24ede-5002-4f02-99c0-9d9eb39b09fc'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:25.495668 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-22d24ede-5002-4f02-99c0-9d9eb39b09fc admin admin] The token cache is being invalidate because project e19e0cdd8e5f4908a00c7807c3635961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:32:25.496827 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-22d24ede-5002-4f02-99c0-9d9eb39b09fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:25.500778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-22d24ede-5002-4f02-99c0-9d9eb39b09fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:25.503422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1141/2311] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:32:25 2022] DELETE /identity/v3/projects/e19e0cdd8e5f4908a00c7807c3635961 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:32:25.515937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42480e-be3f-4e34-8504-0662072801ab None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:25.515937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42480e-be3f-4e34-8504-0662072801ab None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:25.515937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42480e-be3f-4e34-8504-0662072801ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:25.539130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-0b42480e-be3f-4e34-8504-0662072801ab None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:25.540365 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-0b42480e-be3f-4e34-8504-0662072801ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:25.619165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1171/2312] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:32:25.627038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af89b1-a2ba-402b-b100-b21e5d55a83e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:25.627246 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af89b1-a2ba-402b-b100-b21e5d55a83e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:25.627867 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af89b1-a2ba-402b-b100-b21e5d55a83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:25.656667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-69af89b1-a2ba-402b-b100-b21e5d55a83e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:25.658026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-69af89b1-a2ba-402b-b100-b21e5d55a83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:25.667718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1142/2313] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:25.671567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42480e-be3f-4e34-8504-0662072801ab None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:25.687777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42480e-be3f-4e34-8504-0662072801ab None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:25.695038 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c4e12-cc7c-4450-860b-0b6941b986df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:25.695994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c4e12-cc7c-4450-860b-0b6941b986df admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:25.696441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c4e12-cc7c-4450-860b-0b6941b986df admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:25.696873 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c4e12-cc7c-4450-860b-0b6941b986df admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:25.698026 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c4e12-cc7c-4450-860b-0b6941b986df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:32:25.698540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c4e12-cc7c-4450-860b-0b6941b986df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:25.700487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c4e12-cc7c-4450-860b-0b6941b986df admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:25.717136 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1172/2314] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:32:25.721609 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af89b1-a2ba-402b-b100-b21e5d55a83e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:25.733242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af89b1-a2ba-402b-b100-b21e5d55a83e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:25.741112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b494d-6b72-4966-b535-aecdfe0f2818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:25.742270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b494d-6b72-4966-b535-aecdfe0f2818 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:25.743228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b494d-6b72-4966-b535-aecdfe0f2818 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:25.743718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b494d-6b72-4966-b535-aecdfe0f2818 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:25.745225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b494d-6b72-4966-b535-aecdfe0f2818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:25.748385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b494d-6b72-4966-b535-aecdfe0f2818 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:25.783249 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1143/2315] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:32:25 2022] POST /identity/v3/projects => generated 407 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:32:25.786520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c4e12-cc7c-4450-860b-0b6941b986df admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:25.800513 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c4e12-cc7c-4450-860b-0b6941b986df admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:25.891929 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f736f1-f5b6-4b94-be84-f615fa5248b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:25.892854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f736f1-f5b6-4b94-be84-f615fa5248b2 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:25.893476 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f736f1-f5b6-4b94-be84-f615fa5248b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:25.893962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f736f1-f5b6-4b94-be84-f615fa5248b2 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:25.894966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f736f1-f5b6-4b94-be84-f615fa5248b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:25.897256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f736f1-f5b6-4b94-be84-f615fa5248b2 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:25.929907 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1173/2316] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:32:25 2022] POST /identity/v3/users => generated 304 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:25.932885 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b494d-6b72-4966-b535-aecdfe0f2818 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:25.943696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b494d-6b72-4966-b535-aecdfe0f2818 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:25.949409 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e9cf3-c27a-433a-b7eb-697c2b852a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:25.950526 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e9cf3-c27a-433a-b7eb-697c2b852a8f admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:25.950813 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e9cf3-c27a-433a-b7eb-697c2b852a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:25.951092 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e9cf3-c27a-433a-b7eb-697c2b852a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:25.951823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e9cf3-c27a-433a-b7eb-697c2b852a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:25.953459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e9cf3-c27a-433a-b7eb-697c2b852a8f admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:25.970869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1144/2317] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:25.974479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f736f1-f5b6-4b94-be84-f615fa5248b2 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:25.985097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f736f1-f5b6-4b94-be84-f615fa5248b2 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:25.990470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23157ae-00d0-4ed9-a559-bbcb55c4a99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:25.991141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23157ae-00d0-4ed9-a559-bbcb55c4a99f admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:25.991418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23157ae-00d0-4ed9-a559-bbcb55c4a99f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:25.991747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23157ae-00d0-4ed9-a559-bbcb55c4a99f admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:25.992521 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23157ae-00d0-4ed9-a559-bbcb55c4a99f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:25.994169 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23157ae-00d0-4ed9-a559-bbcb55c4a99f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:26.005269 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1174/2318] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:32:26.009352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e9cf3-c27a-433a-b7eb-697c2b852a8f admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:26.020812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e9cf3-c27a-433a-b7eb-697c2b852a8f admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:26.026981 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabbf79-0464-46da-a89b-48e4530eed3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:26.027913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabbf79-0464-46da-a89b-48e4530eed3e admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:26.028209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabbf79-0464-46da-a89b-48e4530eed3e admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:26.028659 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabbf79-0464-46da-a89b-48e4530eed3e admin admin] PATH_INFO: `/v3/projects/68fd978dc9b7403594c9ea47d42c0e35/users/e7c4315420ac4237959fda180c0df962/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:26.049243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabbf79-0464-46da-a89b-48e4530eed3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=68fd978dc9b7403594c9ea47d42c0e35, user_id=e7c4315420ac4237959fda180c0df962, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:26.051971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabbf79-0464-46da-a89b-48e4530eed3e admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:26.074475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1145/2319] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:26 2022] PUT /identity/v3/projects/68fd978dc9b7403594c9ea47d42c0e35/users/e7c4315420ac4237959fda180c0df962/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:26.085923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf48215-a588-4215-b534-c340cd4fbf43 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:26.086210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf48215-a588-4215-b534-c340cd4fbf43 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:26.086493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf48215-a588-4215-b534-c340cd4fbf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:26.122439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-ecf48215-a588-4215-b534-c340cd4fbf43 None None] MFA Rules not processed for user `e7c4315420ac4237959fda180c0df962`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:26.123673 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ecf48215-a588-4215-b534-c340cd4fbf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:26.243001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1175/2320] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:26 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:26.243692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabbf79-0464-46da-a89b-48e4530eed3e admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:26.350053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabbf79-0464-46da-a89b-48e4530eed3e admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:26.367157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7211f21-1fa6-4916-8b2d-5e8d95203186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:26.368358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7211f21-1fa6-4916-8b2d-5e8d95203186 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:26.368709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7211f21-1fa6-4916-8b2d-5e8d95203186 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:26.369576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7211f21-1fa6-4916-8b2d-5e8d95203186 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:26.370868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7211f21-1fa6-4916-8b2d-5e8d95203186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:26.373570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7211f21-1fa6-4916-8b2d-5e8d95203186 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:26.439400 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1146/2321] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:32:26 2022] POST /identity/v3/projects => generated 409 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:32:26.442893 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf48215-a588-4215-b534-c340cd4fbf43 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:26.458123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf48215-a588-4215-b534-c340cd4fbf43 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:26.586425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbd8c8-08d3-4596-ae44-1984ea3f40a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:26.587836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbd8c8-08d3-4596-ae44-1984ea3f40a3 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:26.589870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbd8c8-08d3-4596-ae44-1984ea3f40a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:26.590308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbd8c8-08d3-4596-ae44-1984ea3f40a3 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:26.591433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbd8c8-08d3-4596-ae44-1984ea3f40a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:26.593614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbd8c8-08d3-4596-ae44-1984ea3f40a3 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:26.623784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1176/2322] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:32:26 2022] POST /identity/v3/users => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:26.624764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7211f21-1fa6-4916-8b2d-5e8d95203186 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:26.638858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7211f21-1fa6-4916-8b2d-5e8d95203186 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:26.646402 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ba155-f865-4275-8dcd-fda507fa9666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:26.647644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ba155-f865-4275-8dcd-fda507fa9666 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:26.648325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ba155-f865-4275-8dcd-fda507fa9666 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:26.648913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ba155-f865-4275-8dcd-fda507fa9666 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:26.650470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ba155-f865-4275-8dcd-fda507fa9666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:26.653312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ba155-f865-4275-8dcd-fda507fa9666 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:26.678217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1147/2323] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:32:26.682963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbd8c8-08d3-4596-ae44-1984ea3f40a3 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:26.695744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbd8c8-08d3-4596-ae44-1984ea3f40a3 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:26.701722 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41aba7-74cf-497e-9ec4-a6efa53bf175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:26.702625 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41aba7-74cf-497e-9ec4-a6efa53bf175 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:26.702903 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41aba7-74cf-497e-9ec4-a6efa53bf175 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:26.703176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41aba7-74cf-497e-9ec4-a6efa53bf175 admin admin] PATH_INFO: `/v3/projects/3cc4043175c24d17b0e4c405694688ff/users/b5c8e560771d4c12aa4d0fcb6eee40f8/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:26.720424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41aba7-74cf-497e-9ec4-a6efa53bf175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cc4043175c24d17b0e4c405694688ff, user_id=b5c8e560771d4c12aa4d0fcb6eee40f8, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:26.723275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41aba7-74cf-497e-9ec4-a6efa53bf175 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:26.738573 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1177/2324] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:26 2022] PUT /identity/v3/projects/3cc4043175c24d17b0e4c405694688ff/users/b5c8e560771d4c12aa4d0fcb6eee40f8/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:26.742587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ba155-f865-4275-8dcd-fda507fa9666 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:26.753369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ba155-f865-4275-8dcd-fda507fa9666 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:26.823286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e371172-2437-46f2-8116-992e89b0f53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:26.823994 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e371172-2437-46f2-8116-992e89b0f53d admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:26.824277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e371172-2437-46f2-8116-992e89b0f53d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:26.824547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e371172-2437-46f2-8116-992e89b0f53d admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:26.825307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e371172-2437-46f2-8116-992e89b0f53d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:26.826915 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e371172-2437-46f2-8116-992e89b0f53d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:26.837501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1148/2325] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:26.842250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41aba7-74cf-497e-9ec4-a6efa53bf175 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:26.852622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41aba7-74cf-497e-9ec4-a6efa53bf175 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:26.858317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ebc79-0ed7-4a60-a82c-3ca3ba97804c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:26.859219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ebc79-0ed7-4a60-a82c-3ca3ba97804c admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:26.859496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ebc79-0ed7-4a60-a82c-3ca3ba97804c admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:26.859768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ebc79-0ed7-4a60-a82c-3ca3ba97804c admin admin] PATH_INFO: `/v3/projects/3cc4043175c24d17b0e4c405694688ff/users/b5c8e560771d4c12aa4d0fcb6eee40f8/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:26.861781 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ebc79-0ed7-4a60-a82c-3ca3ba97804c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cc4043175c24d17b0e4c405694688ff, user_id=b5c8e560771d4c12aa4d0fcb6eee40f8, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:26.864473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ebc79-0ed7-4a60-a82c-3ca3ba97804c admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:26.876754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1178/2326] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:26 2022] PUT /identity/v3/projects/3cc4043175c24d17b0e4c405694688ff/users/b5c8e560771d4c12aa4d0fcb6eee40f8/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:26.887561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b20e-680f-43f3-b673-ff24eb2eacc7 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:26.888000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b20e-680f-43f3-b673-ff24eb2eacc7 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:26.888344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b20e-680f-43f3-b673-ff24eb2eacc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:26.919149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-e487b20e-680f-43f3-b673-ff24eb2eacc7 None None] MFA Rules not processed for user `b5c8e560771d4c12aa4d0fcb6eee40f8`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:26.920484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e487b20e-680f-43f3-b673-ff24eb2eacc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:26.994151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1149/2327] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:26 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:27.005934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ebc79-0ed7-4a60-a82c-3ca3ba97804c admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:27.007674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b19ebc79-0ed7-4a60-a82c-3ca3ba97804c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:27.075820 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ebc79-0ed7-4a60-a82c-3ca3ba97804c admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:27.080282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e977d-11d2-4589-9141-4d3f4c9f91cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:27.081327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e977d-11d2-4589-9141-4d3f4c9f91cf service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:27.081820 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e977d-11d2-4589-9141-4d3f4c9f91cf service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:27.082231 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e977d-11d2-4589-9141-4d3f4c9f91cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:27.096452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e977d-11d2-4589-9141-4d3f4c9f91cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:27.097917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e977d-11d2-4589-9141-4d3f4c9f91cf service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:27.109810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1179/2328] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:27 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:32:27.285254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b20e-680f-43f3-b673-ff24eb2eacc7 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:27.294786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b20e-680f-43f3-b673-ff24eb2eacc7 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:27.298584 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18056c94-dffc-48a0-8750-1f26358272d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:27.312435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18056c94-dffc-48a0-8750-1f26358272d4 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:27.313009 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18056c94-dffc-48a0-8750-1f26358272d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:27.313342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18056c94-dffc-48a0-8750-1f26358272d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:27.422448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18056c94-dffc-48a0-8750-1f26358272d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:27.423932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18056c94-dffc-48a0-8750-1f26358272d4 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:27.434614 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1150/2329] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:27 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:32:34.786234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e977d-11d2-4589-9141-4d3f4c9f91cf service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:34.848344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e977d-11d2-4589-9141-4d3f4c9f91cf service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:34.852931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:34.854371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:34.854654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:34.854925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] PATH_INFO: `/v3/users/e7c4315420ac4237959fda180c0df962` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:34.856221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c4315420ac4237959fda180c0df962)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:34.858173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:34.916055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c4315420ac4237959fda180c0df962'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:34.916678 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c4315420ac4237959fda180c0df962'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:34.923671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c4315420ac4237959fda180c0df962'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:34.930566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c4315420ac4237959fda180c0df962'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:34.931382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c4315420ac4237959fda180c0df962'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:34.949130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1180/2330] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:32:34 2022] DELETE /identity/v3/users/e7c4315420ac4237959fda180c0df962 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:34.953935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e6ec1-4f0d-49a8-b205-63f6a6cd5cdd None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:34.954459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e6ec1-4f0d-49a8-b205-63f6a6cd5cdd None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:34.954565 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e6ec1-4f0d-49a8-b205-63f6a6cd5cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:34.972603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-369e6ec1-4f0d-49a8-b205-63f6a6cd5cdd None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:34.973802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-369e6ec1-4f0d-49a8-b205-63f6a6cd5cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:35.032982 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1151/2331] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:35.043540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:35.045008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:35.108305 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676b94-ac35-413d-bbe1-f30f46906f46 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:35.112146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24586509-378c-4316-b63a-c20266563e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:35.112885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24586509-378c-4316-b63a-c20266563e70 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:35.113237 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24586509-378c-4316-b63a-c20266563e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:35.113579 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24586509-378c-4316-b63a-c20266563e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:35.123132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24586509-378c-4316-b63a-c20266563e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:35.124555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24586509-378c-4316-b63a-c20266563e70 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:35.134046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1181/2332] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:32:35.523359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e6ec1-4f0d-49a8-b205-63f6a6cd5cdd None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:35.524921 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-369e6ec1-4f0d-49a8-b205-63f6a6cd5cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:35.537882 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e6ec1-4f0d-49a8-b205-63f6a6cd5cdd None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:35.543284 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:35.544179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:35.544456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:35.544909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] PATH_INFO: `/v3/projects/68fd978dc9b7403594c9ea47d42c0e35` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:35.546110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68fd978dc9b7403594c9ea47d42c0e35)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:35.548112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:35.593513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68fd978dc9b7403594c9ea47d42c0e35', 'request_id': 'req-930355f2-de5b-4484-b8b7-2d1be1b53445'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:35.599374 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68fd978dc9b7403594c9ea47d42c0e35', 'request_id': 'req-930355f2-de5b-4484-b8b7-2d1be1b53445'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:35.605377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68fd978dc9b7403594c9ea47d42c0e35', 'request_id': 'req-930355f2-de5b-4484-b8b7-2d1be1b53445'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:35.616890 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] The token cache is being invalidate because project 68fd978dc9b7403594c9ea47d42c0e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:32:35.617170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:35.617769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:35.620091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1152/2333] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:32:35 2022] DELETE /identity/v3/projects/68fd978dc9b7403594c9ea47d42c0e35 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:35.624061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24586509-378c-4316-b63a-c20266563e70 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:35.633560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24586509-378c-4316-b63a-c20266563e70 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:35.693420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bdf63-3a10-4f48-aa32-35b4324242e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:35.694207 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bdf63-3a10-4f48-aa32-35b4324242e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:35.694487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bdf63-3a10-4f48-aa32-35b4324242e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:35.694759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bdf63-3a10-4f48-aa32-35b4324242e9 admin admin] PATH_INFO: `/v3/users/b5c8e560771d4c12aa4d0fcb6eee40f8` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:35.695966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bdf63-3a10-4f48-aa32-35b4324242e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5c8e560771d4c12aa4d0fcb6eee40f8)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:35.697887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bdf63-3a10-4f48-aa32-35b4324242e9 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:35.741216 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d47bdf63-3a10-4f48-aa32-35b4324242e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5c8e560771d4c12aa4d0fcb6eee40f8'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:35.741870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d47bdf63-3a10-4f48-aa32-35b4324242e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5c8e560771d4c12aa4d0fcb6eee40f8'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:35.749544 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d47bdf63-3a10-4f48-aa32-35b4324242e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5c8e560771d4c12aa4d0fcb6eee40f8'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:35.756255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d47bdf63-3a10-4f48-aa32-35b4324242e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5c8e560771d4c12aa4d0fcb6eee40f8'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:35.756896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d47bdf63-3a10-4f48-aa32-35b4324242e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5c8e560771d4c12aa4d0fcb6eee40f8'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:35.773915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1182/2334] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:32:35 2022] DELETE /identity/v3/users/b5c8e560771d4c12aa4d0fcb6eee40f8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:36.269972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:36.271539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:36.284336 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930355f2-de5b-4484-b8b7-2d1be1b53445 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:36.349441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df98cc7-2653-4999-ae67-a2214daa0108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:36.350549 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df98cc7-2653-4999-ae67-a2214daa0108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:36.351042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df98cc7-2653-4999-ae67-a2214daa0108 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:36.351321 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df98cc7-2653-4999-ae67-a2214daa0108 admin admin] PATH_INFO: `/v3/projects/3cc4043175c24d17b0e4c405694688ff` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:36.352558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df98cc7-2653-4999-ae67-a2214daa0108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cc4043175c24d17b0e4c405694688ff)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:36.355328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df98cc7-2653-4999-ae67-a2214daa0108 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:36.402073 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1df98cc7-2653-4999-ae67-a2214daa0108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cc4043175c24d17b0e4c405694688ff', 'request_id': 'req-1df98cc7-2653-4999-ae67-a2214daa0108'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:36.408182 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1df98cc7-2653-4999-ae67-a2214daa0108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cc4043175c24d17b0e4c405694688ff', 'request_id': 'req-1df98cc7-2653-4999-ae67-a2214daa0108'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:36.413401 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1df98cc7-2653-4999-ae67-a2214daa0108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cc4043175c24d17b0e4c405694688ff', 'request_id': 'req-1df98cc7-2653-4999-ae67-a2214daa0108'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:36.426837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1df98cc7-2653-4999-ae67-a2214daa0108 admin admin] The token cache is being invalidate because project 3cc4043175c24d17b0e4c405694688ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:32:36.427122 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1df98cc7-2653-4999-ae67-a2214daa0108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:36.427923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1df98cc7-2653-4999-ae67-a2214daa0108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:36.429498 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1153/2335] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:32:36 2022] DELETE /identity/v3/projects/3cc4043175c24d17b0e4c405694688ff => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:36.435283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411a5ea-e1f6-4fd7-a582-14f4cace8c8a None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:36.435824 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411a5ea-e1f6-4fd7-a582-14f4cace8c8a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:36.436016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411a5ea-e1f6-4fd7-a582-14f4cace8c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:36.454438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-1411a5ea-e1f6-4fd7-a582-14f4cace8c8a None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:36.455639 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1411a5ea-e1f6-4fd7-a582-14f4cace8c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:36.512567 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10445683-7fc9-4ac5-9b6d-0df2b3924b43 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:36.512934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10445683-7fc9-4ac5-9b6d-0df2b3924b43 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:36.513282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10445683-7fc9-4ac5-9b6d-0df2b3924b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:36.525359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1183/2336] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:36.526976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb1887-fd49-4111-9d57-5f006f340ea1 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:36.527436 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb1887-fd49-4111-9d57-5f006f340ea1 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:36.527700 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb1887-fd49-4111-9d57-5f006f340ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:36.533804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-10445683-7fc9-4ac5-9b6d-0df2b3924b43 None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:36.535107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-10445683-7fc9-4ac5-9b6d-0df2b3924b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:36.547376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-6fcb1887-fd49-4111-9d57-5f006f340ea1 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:36.548696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6fcb1887-fd49-4111-9d57-5f006f340ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:36.559139 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1184/2337] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:32:36.563004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb1887-fd49-4111-9d57-5f006f340ea1 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:36.571957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb1887-fd49-4111-9d57-5f006f340ea1 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:36.576879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28286991-597c-4649-a94d-acab7be6fa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:36.577562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28286991-597c-4649-a94d-acab7be6fa6f admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:36.578600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28286991-597c-4649-a94d-acab7be6fa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:36.578887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28286991-597c-4649-a94d-acab7be6fa6f admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:36.579640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28286991-597c-4649-a94d-acab7be6fa6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:32:36.579996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28286991-597c-4649-a94d-acab7be6fa6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:36.581596 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28286991-597c-4649-a94d-acab7be6fa6f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:36.593968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1185/2338] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:32:36.598946 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28286991-597c-4649-a94d-acab7be6fa6f admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:36.602774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1154/2339] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:32:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:36.609835 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28286991-597c-4649-a94d-acab7be6fa6f admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:36.614492 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45429a1-0b18-48f9-9183-edc229dc02d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:36.615152 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45429a1-0b18-48f9-9183-edc229dc02d7 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:36.615434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45429a1-0b18-48f9-9183-edc229dc02d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:36.615712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45429a1-0b18-48f9-9183-edc229dc02d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:36.616575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45429a1-0b18-48f9-9183-edc229dc02d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:36.618633 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45429a1-0b18-48f9-9183-edc229dc02d7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:36.620519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10445683-7fc9-4ac5-9b6d-0df2b3924b43 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:36.621941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-10445683-7fc9-4ac5-9b6d-0df2b3924b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:36.658086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1186/2340] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:32:36 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:32:36.661864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45429a1-0b18-48f9-9183-edc229dc02d7 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:36.696174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10445683-7fc9-4ac5-9b6d-0df2b3924b43 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:36.723000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45429a1-0b18-48f9-9183-edc229dc02d7 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:36.727779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddafdf6-a6fe-4827-af9b-fd68f1afe12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:36.729492 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddafdf6-a6fe-4827-af9b-fd68f1afe12f admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:36.729952 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddafdf6-a6fe-4827-af9b-fd68f1afe12f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:36.730326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddafdf6-a6fe-4827-af9b-fd68f1afe12f admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:36.731067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddafdf6-a6fe-4827-af9b-fd68f1afe12f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:36.733141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddafdf6-a6fe-4827-af9b-fd68f1afe12f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:36.757339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1187/2341] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:32:36 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:32:36.761164 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949600b7-06e8-476a-80f1-83d11f2df376 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:36.761944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddafdf6-a6fe-4827-af9b-fd68f1afe12f admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:36.762270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949600b7-06e8-476a-80f1-83d11f2df376 service swift] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:36.762799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949600b7-06e8-476a-80f1-83d11f2df376 service swift] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:36.763307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949600b7-06e8-476a-80f1-83d11f2df376 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:36.774283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddafdf6-a6fe-4827-af9b-fd68f1afe12f admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:36.781085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f03ba-23da-4bfe-abc0-7c6efdeb8e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:36.782030 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f03ba-23da-4bfe-abc0-7c6efdeb8e62 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:36.782384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f03ba-23da-4bfe-abc0-7c6efdeb8e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:36.782726 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f03ba-23da-4bfe-abc0-7c6efdeb8e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:36.783537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f03ba-23da-4bfe-abc0-7c6efdeb8e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:36.785456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f03ba-23da-4bfe-abc0-7c6efdeb8e62 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:36.797884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1188/2342] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:36.801806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f03ba-23da-4bfe-abc0-7c6efdeb8e62 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:36.813699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f03ba-23da-4bfe-abc0-7c6efdeb8e62 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:36.819664 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1066dcd-d995-44bb-8033-c783f4777bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:36.820562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1066dcd-d995-44bb-8033-c783f4777bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:36.821580 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1066dcd-d995-44bb-8033-c783f4777bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:36.822090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1066dcd-d995-44bb-8033-c783f4777bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:36.823693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1066dcd-d995-44bb-8033-c783f4777bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:36.825496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1066dcd-d995-44bb-8033-c783f4777bc6 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:36.827037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949600b7-06e8-476a-80f1-83d11f2df376 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:36.828777 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949600b7-06e8-476a-80f1-83d11f2df376 service swift] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:36.839253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1189/2343] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:36.839868 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1155/2344] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:32:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 09 04:32:36.843951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949600b7-06e8-476a-80f1-83d11f2df376 service swift] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:36.855844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949600b7-06e8-476a-80f1-83d11f2df376 service swift] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:36.861624 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cee2a4-6ff6-4b81-b74b-8b8c60427477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:36.862801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cee2a4-6ff6-4b81-b74b-8b8c60427477 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:36.863098 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cee2a4-6ff6-4b81-b74b-8b8c60427477 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:36.863378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cee2a4-6ff6-4b81-b74b-8b8c60427477 admin admin] PATH_INFO: `/v3/projects/f91e0bacc98046d5bdc54fc8308a36ef/users/7a4e196ee2db4410abfb921da249b3df/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:36.882886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cee2a4-6ff6-4b81-b74b-8b8c60427477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f91e0bacc98046d5bdc54fc8308a36ef, user_id=7a4e196ee2db4410abfb921da249b3df, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:36.886432 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cee2a4-6ff6-4b81-b74b-8b8c60427477 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:36.912333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1156/2345] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:36 2022] PUT /identity/v3/projects/f91e0bacc98046d5bdc54fc8308a36ef/users/7a4e196ee2db4410abfb921da249b3df/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:36.921784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c7a22-b582-4d05-bca0-e7bc1acb9680 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:36.922135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c7a22-b582-4d05-bca0-e7bc1acb9680 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:36.922451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c7a22-b582-4d05-bca0-e7bc1acb9680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:36.981330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-b23c7a22-b582-4d05-bca0-e7bc1acb9680 None None] MFA Rules not processed for user `7a4e196ee2db4410abfb921da249b3df`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:36.982671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b23c7a22-b582-4d05-bca0-e7bc1acb9680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:37.111350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1190/2346] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:37.112423 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cee2a4-6ff6-4b81-b74b-8b8c60427477 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:37.217586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cee2a4-6ff6-4b81-b74b-8b8c60427477 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:37.224344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61963f6-8615-4374-9de1-7e7edc05e2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:37.225461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61963f6-8615-4374-9de1-7e7edc05e2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:37.225934 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61963f6-8615-4374-9de1-7e7edc05e2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:37.226308 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61963f6-8615-4374-9de1-7e7edc05e2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:37.227741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61963f6-8615-4374-9de1-7e7edc05e2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:37.231147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61963f6-8615-4374-9de1-7e7edc05e2b6 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:37.302061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c7a22-b582-4d05-bca0-e7bc1acb9680 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:37.304794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1157/2347] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:32:37 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:32:37.322567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c7a22-b582-4d05-bca0-e7bc1acb9680 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:37.430446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee8b0b-73d1-4998-aee1-836ae35b73e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:37.431814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8b0b-73d1-4998-aee1-836ae35b73e1 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:37.432622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8b0b-73d1-4998-aee1-836ae35b73e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:37.433301 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8b0b-73d1-4998-aee1-836ae35b73e1 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:37.434469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee8b0b-73d1-4998-aee1-836ae35b73e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:37.436560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee8b0b-73d1-4998-aee1-836ae35b73e1 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:37.490057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1191/2348] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:32:37 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:37.495294 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61963f6-8615-4374-9de1-7e7edc05e2b6 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:37.521292 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61963f6-8615-4374-9de1-7e7edc05e2b6 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:37.527346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495a5ca-cc45-48be-9fd7-71c1826ce82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:37.528761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495a5ca-cc45-48be-9fd7-71c1826ce82d admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:37.529840 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495a5ca-cc45-48be-9fd7-71c1826ce82d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:37.530530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495a5ca-cc45-48be-9fd7-71c1826ce82d admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:37.531702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495a5ca-cc45-48be-9fd7-71c1826ce82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:37.534272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495a5ca-cc45-48be-9fd7-71c1826ce82d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:37.556180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1158/2349] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:37.561706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee8b0b-73d1-4998-aee1-836ae35b73e1 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:37.581994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee8b0b-73d1-4998-aee1-836ae35b73e1 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:37.591925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5ceea-12c3-46df-8595-d65a8df383e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:37.593585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5ceea-12c3-46df-8595-d65a8df383e4 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:37.594295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5ceea-12c3-46df-8595-d65a8df383e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:37.595086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5ceea-12c3-46df-8595-d65a8df383e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:37.596422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5ceea-12c3-46df-8595-d65a8df383e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:37.600349 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5ceea-12c3-46df-8595-d65a8df383e4 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:37.623168 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1192/2350] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:37.627796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495a5ca-cc45-48be-9fd7-71c1826ce82d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:37.643167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495a5ca-cc45-48be-9fd7-71c1826ce82d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:37.654882 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2168ec-cf50-4d1f-afae-7980ddc93827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:37.657524 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2168ec-cf50-4d1f-afae-7980ddc93827 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:37.658000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2168ec-cf50-4d1f-afae-7980ddc93827 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:37.658566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2168ec-cf50-4d1f-afae-7980ddc93827 admin admin] PATH_INFO: `/v3/projects/7fe8be6d70ad498ca90790fa3f995eb0/users/e7fc756456024203966499d3797aa20a/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:37.677907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2168ec-cf50-4d1f-afae-7980ddc93827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fe8be6d70ad498ca90790fa3f995eb0, user_id=e7fc756456024203966499d3797aa20a, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:37.680909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2168ec-cf50-4d1f-afae-7980ddc93827 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:37.718077 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1159/2351] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:37 2022] PUT /identity/v3/projects/7fe8be6d70ad498ca90790fa3f995eb0/users/e7fc756456024203966499d3797aa20a/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:37.730467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9244c8-2d44-4618-95ff-4547f81d84e2 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:37.730764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9244c8-2d44-4618-95ff-4547f81d84e2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:37.731043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9244c8-2d44-4618-95ff-4547f81d84e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:37.781456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-ec9244c8-2d44-4618-95ff-4547f81d84e2 None None] MFA Rules not processed for user `e7fc756456024203966499d3797aa20a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:37.783123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ec9244c8-2d44-4618-95ff-4547f81d84e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:37.864687 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1193/2352] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:37.869541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2168ec-cf50-4d1f-afae-7980ddc93827 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:37.961141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2168ec-cf50-4d1f-afae-7980ddc93827 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:37.969360 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada645a2-32c6-41f4-9214-86bc3e9d4ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:37.970021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada645a2-32c6-41f4-9214-86bc3e9d4ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:37.970311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada645a2-32c6-41f4-9214-86bc3e9d4ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:37.970688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada645a2-32c6-41f4-9214-86bc3e9d4ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:37.971300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada645a2-32c6-41f4-9214-86bc3e9d4ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:37.973456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada645a2-32c6-41f4-9214-86bc3e9d4ea7 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:38.006773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1160/2353] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:32:37 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:38.011324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9244c8-2d44-4618-95ff-4547f81d84e2 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:38.030727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9244c8-2d44-4618-95ff-4547f81d84e2 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:38.105185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a83a0a-c734-4cac-8e00-040ad86c797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:38.105185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a83a0a-c734-4cac-8e00-040ad86c797a admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:38.105740 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a83a0a-c734-4cac-8e00-040ad86c797a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:38.106092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a83a0a-c734-4cac-8e00-040ad86c797a admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:38.106873 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a83a0a-c734-4cac-8e00-040ad86c797a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:38.108910 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a83a0a-c734-4cac-8e00-040ad86c797a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:38.141080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada645a2-32c6-41f4-9214-86bc3e9d4ea7 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:38.142801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-ada645a2-32c6-41f4-9214-86bc3e9d4ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:38.154812 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1194/2354] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:32:38 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:38.159629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a83a0a-c734-4cac-8e00-040ad86c797a admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:38.180983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a83a0a-c734-4cac-8e00-040ad86c797a admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:38.196135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d958e-6fa0-4e86-9789-25e1da136ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:38.196846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d958e-6fa0-4e86-9789-25e1da136ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:38.197147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d958e-6fa0-4e86-9789-25e1da136ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:38.198101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d958e-6fa0-4e86-9789-25e1da136ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:38.198852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d958e-6fa0-4e86-9789-25e1da136ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:38.203703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d958e-6fa0-4e86-9789-25e1da136ea1 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:38.225165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1195/2355] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:38.231402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d958e-6fa0-4e86-9789-25e1da136ea1 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:38.253006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d958e-6fa0-4e86-9789-25e1da136ea1 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:38.261094 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7203a3-1dae-4682-9671-0d6ca26947bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:38.262512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7203a3-1dae-4682-9671-0d6ca26947bc admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:38.263126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7203a3-1dae-4682-9671-0d6ca26947bc admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:38.263597 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7203a3-1dae-4682-9671-0d6ca26947bc admin admin] PATH_INFO: `/v3/projects/5eb39ee0d4374240b7fd29e0bf23c970/users/28dc85dac8b24be0bc4fcbb9faa6b8e0/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:38.283173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada645a2-32c6-41f4-9214-86bc3e9d4ea7 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:38.288913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dfb9a-ae81-48f3-bed8-50513cd04fac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:38.289675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dfb9a-ae81-48f3-bed8-50513cd04fac service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:38.290040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dfb9a-ae81-48f3-bed8-50513cd04fac service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:38.290388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dfb9a-ae81-48f3-bed8-50513cd04fac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:38.294067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-eb3dfb9a-ae81-48f3-bed8-50513cd04fac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:38.311332 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7203a3-1dae-4682-9671-0d6ca26947bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eb39ee0d4374240b7fd29e0bf23c970, user_id=28dc85dac8b24be0bc4fcbb9faa6b8e0, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:38.311634 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3dfb9a-ae81-48f3-bed8-50513cd04fac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:38.313690 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3dfb9a-ae81-48f3-bed8-50513cd04fac service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:38.321576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7203a3-1dae-4682-9671-0d6ca26947bc admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:38.326715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1161/2356] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:32:38.347661 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1196/2357] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:38 2022] PUT /identity/v3/projects/5eb39ee0d4374240b7fd29e0bf23c970/users/28dc85dac8b24be0bc4fcbb9faa6b8e0/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:38.352645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dfb9a-ae81-48f3-bed8-50513cd04fac service nova] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:38.373519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dfb9a-ae81-48f3-bed8-50513cd04fac service nova] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:38.452999 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa581f7-0a0a-434b-b50f-2fcbcb7efd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:38.453877 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa581f7-0a0a-434b-b50f-2fcbcb7efd0c admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:38.454263 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa581f7-0a0a-434b-b50f-2fcbcb7efd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:38.454647 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa581f7-0a0a-434b-b50f-2fcbcb7efd0c admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:38.455491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa581f7-0a0a-434b-b50f-2fcbcb7efd0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:38.457232 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa581f7-0a0a-434b-b50f-2fcbcb7efd0c admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:38.474532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7203a3-1dae-4682-9671-0d6ca26947bc admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:38.475092 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1162/2358] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:32:38.487422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7203a3-1dae-4682-9671-0d6ca26947bc admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:38.496895 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075d79a-d6e5-4f65-b6c4-cdd385629319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:38.497807 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075d79a-d6e5-4f65-b6c4-cdd385629319 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:38.498413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075d79a-d6e5-4f65-b6c4-cdd385629319 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:38.498413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075d79a-d6e5-4f65-b6c4-cdd385629319 admin admin] PATH_INFO: `/v3/projects/5eb39ee0d4374240b7fd29e0bf23c970/users/28dc85dac8b24be0bc4fcbb9faa6b8e0/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:38.500424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075d79a-d6e5-4f65-b6c4-cdd385629319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eb39ee0d4374240b7fd29e0bf23c970, user_id=28dc85dac8b24be0bc4fcbb9faa6b8e0, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:38.503056 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075d79a-d6e5-4f65-b6c4-cdd385629319 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:38.516786 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1197/2359] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:38 2022] PUT /identity/v3/projects/5eb39ee0d4374240b7fd29e0bf23c970/users/28dc85dac8b24be0bc4fcbb9faa6b8e0/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:38.527038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d5e3e-f9d7-4236-94f1-426ca6865416 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:38.527325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d5e3e-f9d7-4236-94f1-426ca6865416 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:38.527604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d5e3e-f9d7-4236-94f1-426ca6865416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:38.582844 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-8b6d5e3e-f9d7-4236-94f1-426ca6865416 None None] MFA Rules not processed for user `28dc85dac8b24be0bc4fcbb9faa6b8e0`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:38.584069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8b6d5e3e-f9d7-4236-94f1-426ca6865416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:38.721921 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1163/2360] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:38.734153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075d79a-d6e5-4f65-b6c4-cdd385629319 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:38.736847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8075d79a-d6e5-4f65-b6c4-cdd385629319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:38.864875 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075d79a-d6e5-4f65-b6c4-cdd385629319 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:38.868654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382652a-9d12-44fc-86e2-8c2a90b9aaeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:38.869923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382652a-9d12-44fc-86e2-8c2a90b9aaeb service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:38.870239 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382652a-9d12-44fc-86e2-8c2a90b9aaeb service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:38.870624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382652a-9d12-44fc-86e2-8c2a90b9aaeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:38.882603 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382652a-9d12-44fc-86e2-8c2a90b9aaeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:38.885409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382652a-9d12-44fc-86e2-8c2a90b9aaeb service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:38.909295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1198/2361] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:32:38.955735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d5e3e-f9d7-4236-94f1-426ca6865416 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:38.960976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8b6d5e3e-f9d7-4236-94f1-426ca6865416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:38.985416 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d5e3e-f9d7-4236-94f1-426ca6865416 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:38.989772 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0960b4-80e0-454c-a831-77c921fc9e2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:38.991012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0960b4-80e0-454c-a831-77c921fc9e2c service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:38.991300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0960b4-80e0-454c-a831-77c921fc9e2c service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:38.991578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0960b4-80e0-454c-a831-77c921fc9e2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:39.090771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0960b4-80e0-454c-a831-77c921fc9e2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:39.092408 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0960b4-80e0-454c-a831-77c921fc9e2c service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:39.102837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1164/2362] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:32:42.110463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382652a-9d12-44fc-86e2-8c2a90b9aaeb service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:42.176758 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382652a-9d12-44fc-86e2-8c2a90b9aaeb service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:42.180445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:42.181354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:42.181688 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:42.182138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] PATH_INFO: `/v3/users/7a4e196ee2db4410abfb921da249b3df` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:42.183349 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a4e196ee2db4410abfb921da249b3df)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:42.185328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:42.254775 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a4e196ee2db4410abfb921da249b3df'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:42.255479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a4e196ee2db4410abfb921da249b3df'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:42.266218 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a4e196ee2db4410abfb921da249b3df'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:42.273921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a4e196ee2db4410abfb921da249b3df'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:42.275614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a4e196ee2db4410abfb921da249b3df'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:42.308360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1199/2363] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:32:42 2022] DELETE /identity/v3/users/7a4e196ee2db4410abfb921da249b3df => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:42.312775 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bec9b-cd1c-4b92-acc1-df1280725c34 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:42.313252 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bec9b-cd1c-4b92-acc1-df1280725c34 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:42.313458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bec9b-cd1c-4b92-acc1-df1280725c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:42.332327 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-0e3bec9b-cd1c-4b92-acc1-df1280725c34 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:42.333040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0e3bec9b-cd1c-4b92-acc1-df1280725c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:42.419014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1165/2364] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:42.435370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:42.442331 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:42.511298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5c152-314b-4b16-9f7d-a54645f01673 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:42.514962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20533d91-42f5-4be9-a161-ba5b0e73b1c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:42.515609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20533d91-42f5-4be9-a161-ba5b0e73b1c7 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:42.515886 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20533d91-42f5-4be9-a161-ba5b0e73b1c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:42.516426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20533d91-42f5-4be9-a161-ba5b0e73b1c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:42.527737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20533d91-42f5-4be9-a161-ba5b0e73b1c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:42.529222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20533d91-42f5-4be9-a161-ba5b0e73b1c7 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:42.539182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1200/2365] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:32:42.847814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bec9b-cd1c-4b92-acc1-df1280725c34 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:42.849586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0e3bec9b-cd1c-4b92-acc1-df1280725c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:42.861997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bec9b-cd1c-4b92-acc1-df1280725c34 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:42.870022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:42.871027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:42.871304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:42.871683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] PATH_INFO: `/v3/projects/f91e0bacc98046d5bdc54fc8308a36ef` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:42.876832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f91e0bacc98046d5bdc54fc8308a36ef)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:42.879195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:42.937570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f91e0bacc98046d5bdc54fc8308a36ef', 'request_id': 'req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:42.945129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f91e0bacc98046d5bdc54fc8308a36ef', 'request_id': 'req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:42.951001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f91e0bacc98046d5bdc54fc8308a36ef', 'request_id': 'req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:42.963272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] The token cache is being invalidate because project f91e0bacc98046d5bdc54fc8308a36ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:32:42.963874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:42.964501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:42.969129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1166/2366] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:32:42 2022] DELETE /identity/v3/projects/f91e0bacc98046d5bdc54fc8308a36ef => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:42.970110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20533d91-42f5-4be9-a161-ba5b0e73b1c7 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:42.982634 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20533d91-42f5-4be9-a161-ba5b0e73b1c7 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:43.043275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:43.044224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:43.044507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:43.044889 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] PATH_INFO: `/v3/users/e7fc756456024203966499d3797aa20a` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:43.048400 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7fc756456024203966499d3797aa20a)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:43.050450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:43.099166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7fc756456024203966499d3797aa20a'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.099904 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7fc756456024203966499d3797aa20a'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.117181 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7fc756456024203966499d3797aa20a'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.126117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7fc756456024203966499d3797aa20a'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.126819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7fc756456024203966499d3797aa20a'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.144644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1201/2367] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:32:42 2022] DELETE /identity/v3/users/e7fc756456024203966499d3797aa20a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:32:43.460568 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:43.463206 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:43.477817 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cbe1c-3e69-4eb8-af11-7dd8a64d71f9 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:43.562507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:43.563635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:43.564003 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:43.564430 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] PATH_INFO: `/v3/projects/7fe8be6d70ad498ca90790fa3f995eb0` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:43.565824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fe8be6d70ad498ca90790fa3f995eb0)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:43.567814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:43.625516 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fe8be6d70ad498ca90790fa3f995eb0', 'request_id': 'req-72ea0a55-4634-40bc-b076-87024fa0b436'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.632426 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fe8be6d70ad498ca90790fa3f995eb0', 'request_id': 'req-72ea0a55-4634-40bc-b076-87024fa0b436'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.638595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fe8be6d70ad498ca90790fa3f995eb0', 'request_id': 'req-72ea0a55-4634-40bc-b076-87024fa0b436'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.669013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] The token cache is being invalidate because project 7fe8be6d70ad498ca90790fa3f995eb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:32:43.669310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.669966 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.671582 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1167/2368] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:32:43 2022] DELETE /identity/v3/projects/7fe8be6d70ad498ca90790fa3f995eb0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:43.677927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:43.680270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:43.695739 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a1a9b-f73f-42d0-82bd-055da9c57c55 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:43.771520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bf5c3-f207-43cc-a1a2-984b82f2d0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:43.772836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bf5c3-f207-43cc-a1a2-984b82f2d0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:43.773290 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bf5c3-f207-43cc-a1a2-984b82f2d0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:43.773720 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bf5c3-f207-43cc-a1a2-984b82f2d0d9 admin admin] PATH_INFO: `/v3/users/28dc85dac8b24be0bc4fcbb9faa6b8e0` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:43.775840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bf5c3-f207-43cc-a1a2-984b82f2d0d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28dc85dac8b24be0bc4fcbb9faa6b8e0)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:43.778097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bf5c3-f207-43cc-a1a2-984b82f2d0d9 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:43.831821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4d1bf5c3-f207-43cc-a1a2-984b82f2d0d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28dc85dac8b24be0bc4fcbb9faa6b8e0'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.832456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4d1bf5c3-f207-43cc-a1a2-984b82f2d0d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28dc85dac8b24be0bc4fcbb9faa6b8e0'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.842199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4d1bf5c3-f207-43cc-a1a2-984b82f2d0d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28dc85dac8b24be0bc4fcbb9faa6b8e0'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.849675 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4d1bf5c3-f207-43cc-a1a2-984b82f2d0d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28dc85dac8b24be0bc4fcbb9faa6b8e0'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.850378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4d1bf5c3-f207-43cc-a1a2-984b82f2d0d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28dc85dac8b24be0bc4fcbb9faa6b8e0'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:43.871841 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1202/2369] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:32:43 2022] DELETE /identity/v3/users/28dc85dac8b24be0bc4fcbb9faa6b8e0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:44.192694 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:44.194347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:44.216609 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea0a55-4634-40bc-b076-87024fa0b436 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:44.275209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2033e-e1ba-447f-b87f-d4261bc79415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:44.276110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2033e-e1ba-447f-b87f-d4261bc79415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:44.276391 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2033e-e1ba-447f-b87f-d4261bc79415 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:44.276660 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2033e-e1ba-447f-b87f-d4261bc79415 admin admin] PATH_INFO: `/v3/projects/5eb39ee0d4374240b7fd29e0bf23c970` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:44.277869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2033e-e1ba-447f-b87f-d4261bc79415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eb39ee0d4374240b7fd29e0bf23c970)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:44.279763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2033e-e1ba-447f-b87f-d4261bc79415 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:44.328148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-04d2033e-e1ba-447f-b87f-d4261bc79415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eb39ee0d4374240b7fd29e0bf23c970', 'request_id': 'req-04d2033e-e1ba-447f-b87f-d4261bc79415'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:44.334481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-04d2033e-e1ba-447f-b87f-d4261bc79415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eb39ee0d4374240b7fd29e0bf23c970', 'request_id': 'req-04d2033e-e1ba-447f-b87f-d4261bc79415'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:44.349778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-04d2033e-e1ba-447f-b87f-d4261bc79415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eb39ee0d4374240b7fd29e0bf23c970', 'request_id': 'req-04d2033e-e1ba-447f-b87f-d4261bc79415'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:44.369756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-04d2033e-e1ba-447f-b87f-d4261bc79415 admin admin] The token cache is being invalidate because project 5eb39ee0d4374240b7fd29e0bf23c970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:32:44.370041 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-04d2033e-e1ba-447f-b87f-d4261bc79415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:44.370832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-04d2033e-e1ba-447f-b87f-d4261bc79415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:32:44.373456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1168/2370] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:32:44 2022] DELETE /identity/v3/projects/5eb39ee0d4374240b7fd29e0bf23c970 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:44.380294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc025e7a-c58e-44dc-8ce2-78a79039982b None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:44.381125 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc025e7a-c58e-44dc-8ce2-78a79039982b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:44.381274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc025e7a-c58e-44dc-8ce2-78a79039982b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:44.403484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-fc025e7a-c58e-44dc-8ce2-78a79039982b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:44.404632 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-fc025e7a-c58e-44dc-8ce2-78a79039982b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:44.481942 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1203/2371] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:44.490648 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e268d2-3a25-4ab8-99b7-92a9127acb9e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:44.491231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e268d2-3a25-4ab8-99b7-92a9127acb9e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:44.499255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e268d2-3a25-4ab8-99b7-92a9127acb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:44.524768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-48e268d2-3a25-4ab8-99b7-92a9127acb9e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:44.526446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-48e268d2-3a25-4ab8-99b7-92a9127acb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:44.539657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1169/2372] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:44.542063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc025e7a-c58e-44dc-8ce2-78a79039982b None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:44.551996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc025e7a-c58e-44dc-8ce2-78a79039982b None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:44.557274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd68a5d-80b9-497c-8917-25fd176f99b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:44.557943 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd68a5d-80b9-497c-8917-25fd176f99b9 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:44.558224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd68a5d-80b9-497c-8917-25fd176f99b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:44.558496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd68a5d-80b9-497c-8917-25fd176f99b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:44.559228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd68a5d-80b9-497c-8917-25fd176f99b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:32:44.559532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd68a5d-80b9-497c-8917-25fd176f99b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:44.561245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd68a5d-80b9-497c-8917-25fd176f99b9 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:44.575939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1204/2373] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:32:44.578454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e268d2-3a25-4ab8-99b7-92a9127acb9e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:44.588426 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e268d2-3a25-4ab8-99b7-92a9127acb9e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:44.593532 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59a702-6b6c-4254-ad33-d37b01592955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:44.594266 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59a702-6b6c-4254-ad33-d37b01592955 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:44.594618 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59a702-6b6c-4254-ad33-d37b01592955 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:44.594958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59a702-6b6c-4254-ad33-d37b01592955 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:44.595716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59a702-6b6c-4254-ad33-d37b01592955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:44.597636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59a702-6b6c-4254-ad33-d37b01592955 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:44.625855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1170/2374] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:32:44 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:44.629593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd68a5d-80b9-497c-8917-25fd176f99b9 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:44.641052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd68a5d-80b9-497c-8917-25fd176f99b9 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:44.713082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48979a0b-708c-449f-935a-75bf8e941d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:44.714156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48979a0b-708c-449f-935a-75bf8e941d85 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:44.714576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48979a0b-708c-449f-935a-75bf8e941d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:44.714836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48979a0b-708c-449f-935a-75bf8e941d85 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:44.715603 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48979a0b-708c-449f-935a-75bf8e941d85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:44.717727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48979a0b-708c-449f-935a-75bf8e941d85 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:44.749332 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1205/2375] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:32:44 2022] POST /identity/v3/users => generated 284 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:32:44.753391 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59a702-6b6c-4254-ad33-d37b01592955 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:44.767185 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59a702-6b6c-4254-ad33-d37b01592955 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:44.776094 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2d4de-5072-4a45-9ac8-223b10551e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:44.776797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2d4de-5072-4a45-9ac8-223b10551e89 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:44.777093 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2d4de-5072-4a45-9ac8-223b10551e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:44.777373 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2d4de-5072-4a45-9ac8-223b10551e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:44.778114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2d4de-5072-4a45-9ac8-223b10551e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:44.779710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2d4de-5072-4a45-9ac8-223b10551e89 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:44.795329 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1171/2376] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:32:44.800958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48979a0b-708c-449f-935a-75bf8e941d85 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:44.814227 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48979a0b-708c-449f-935a-75bf8e941d85 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:44.819849 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ece26-4705-4ceb-ac05-0d8571cf7058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:44.820670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ece26-4705-4ceb-ac05-0d8571cf7058 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:44.820954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ece26-4705-4ceb-ac05-0d8571cf7058 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:44.821299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ece26-4705-4ceb-ac05-0d8571cf7058 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:44.822969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ece26-4705-4ceb-ac05-0d8571cf7058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:44.824692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ece26-4705-4ceb-ac05-0d8571cf7058 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:44.838498 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1206/2377] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:32:44.839638 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2d4de-5072-4a45-9ac8-223b10551e89 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:44.851723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2d4de-5072-4a45-9ac8-223b10551e89 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:44.858856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff25d05-829c-4853-8bc8-403cbfad34d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:44.859952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff25d05-829c-4853-8bc8-403cbfad34d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:44.860365 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff25d05-829c-4853-8bc8-403cbfad34d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:44.860729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff25d05-829c-4853-8bc8-403cbfad34d1 admin admin] PATH_INFO: `/v3/projects/88cc42b2de4448afbfd3c86b2d474793/users/2a08c9e49cfc4a55840f17f1124411c9/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:44.877275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff25d05-829c-4853-8bc8-403cbfad34d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88cc42b2de4448afbfd3c86b2d474793, user_id=2a08c9e49cfc4a55840f17f1124411c9, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:44.880246 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff25d05-829c-4853-8bc8-403cbfad34d1 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:44.896882 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1172/2378] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:32:44 2022] PUT /identity/v3/projects/88cc42b2de4448afbfd3c86b2d474793/users/2a08c9e49cfc4a55840f17f1124411c9/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:32:44.908079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44eeea-3da0-4bd0-99d2-f2a66e21a5ed None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:44.908401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44eeea-3da0-4bd0-99d2-f2a66e21a5ed None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:44.908645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44eeea-3da0-4bd0-99d2-f2a66e21a5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:44.937971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-0a44eeea-3da0-4bd0-99d2-f2a66e21a5ed None None] MFA Rules not processed for user `2a08c9e49cfc4a55840f17f1124411c9`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:44.939165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-0a44eeea-3da0-4bd0-99d2-f2a66e21a5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:45.003081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1207/2379] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:32:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:45.015639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff25d05-829c-4853-8bc8-403cbfad34d1 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:45.017386 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5ff25d05-829c-4853-8bc8-403cbfad34d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:45.096174 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff25d05-829c-4853-8bc8-403cbfad34d1 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:45.099808 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648b623-14b9-4a1d-8495-3b684df14e44 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '295bb46627e043989e6dba2805607c99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:45.100481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648b623-14b9-4a1d-8495-3b684df14e44 service glance] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:45.100770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648b623-14b9-4a1d-8495-3b684df14e44 service glance] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:45.101043 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648b623-14b9-4a1d-8495-3b684df14e44 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:45.109971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e648b623-14b9-4a1d-8495-3b684df14e44 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:45.111377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e648b623-14b9-4a1d-8495-3b684df14e44 service glance] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:45.121117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1173/2380] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:32:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:32:53.520674 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f502ff-7bf4-42c5-8eae-8a47614ece9c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:53.521316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f502ff-7bf4-42c5-8eae-8a47614ece9c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:53.521316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f502ff-7bf4-42c5-8eae-8a47614ece9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:53.541244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c8f502ff-7bf4-42c5-8eae-8a47614ece9c None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:32:53.542556 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c8f502ff-7bf4-42c5-8eae-8a47614ece9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:53.597204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1208/2381] 10.222.0.44 () {58 vars in 967 bytes} [Mon May 9 04:32:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:32:53.621251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648b623-14b9-4a1d-8495-3b684df14e44 service glance] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:32:53.622662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e648b623-14b9-4a1d-8495-3b684df14e44 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:32:53.693097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648b623-14b9-4a1d-8495-3b684df14e44 service glance] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:32:53.698412 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9286bda-93ce-431a-aa2c-eecdc1b6c8a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:32:53.699053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9286bda-93ce-431a-aa2c-eecdc1b6c8a6 service swift] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:32:53.699336 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9286bda-93ce-431a-aa2c-eecdc1b6c8a6 service swift] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:32:53.699609 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9286bda-93ce-431a-aa2c-eecdc1b6c8a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:32:53.707360 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9286bda-93ce-431a-aa2c-eecdc1b6c8a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:32:53.708930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9286bda-93ce-431a-aa2c-eecdc1b6c8a6 service swift] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:32:53.718453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1174/2382] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:32:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:33:13.599980 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f502ff-7bf4-42c5-8eae-8a47614ece9c None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:13.601360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c8f502ff-7bf4-42c5-8eae-8a47614ece9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:13.655723 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f502ff-7bf4-42c5-8eae-8a47614ece9c None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:13.659535 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65303780-faa0-4376-8db2-02e3b09867ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:13.660209 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65303780-faa0-4376-8db2-02e3b09867ba service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:13.660486 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65303780-faa0-4376-8db2-02e3b09867ba service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:13.660755 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65303780-faa0-4376-8db2-02e3b09867ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:13.662482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-65303780-faa0-4376-8db2-02e3b09867ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:13.720988 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65303780-faa0-4376-8db2-02e3b09867ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:13.722392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65303780-faa0-4376-8db2-02e3b09867ba service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:13.731633 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1209/2383] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:16.682288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9286bda-93ce-431a-aa2c-eecdc1b6c8a6 service swift] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:16.683920 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a9286bda-93ce-431a-aa2c-eecdc1b6c8a6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:16.698948 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9286bda-93ce-431a-aa2c-eecdc1b6c8a6 service swift] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:16.704279 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c681b0-cc1b-4dae-a26f-326aad20b4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:16.705148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c681b0-cc1b-4dae-a26f-326aad20b4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:16.705431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c681b0-cc1b-4dae-a26f-326aad20b4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:16.705704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c681b0-cc1b-4dae-a26f-326aad20b4dc admin admin] PATH_INFO: `/v3/users/85429c28c994443eb30b98ece56c4c9b` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:16.706912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c681b0-cc1b-4dae-a26f-326aad20b4dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=85429c28c994443eb30b98ece56c4c9b)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:16.708854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c681b0-cc1b-4dae-a26f-326aad20b4dc admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:16.757859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-53c681b0-cc1b-4dae-a26f-326aad20b4dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85429c28c994443eb30b98ece56c4c9b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:16.758505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-53c681b0-cc1b-4dae-a26f-326aad20b4dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85429c28c994443eb30b98ece56c4c9b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:16.759104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-53c681b0-cc1b-4dae-a26f-326aad20b4dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85429c28c994443eb30b98ece56c4c9b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:16.768002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-53c681b0-cc1b-4dae-a26f-326aad20b4dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85429c28c994443eb30b98ece56c4c9b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:16.774810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-53c681b0-cc1b-4dae-a26f-326aad20b4dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85429c28c994443eb30b98ece56c4c9b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:16.792200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1175/2384] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:16 2022] DELETE /identity/v3/users/85429c28c994443eb30b98ece56c4c9b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:16.963834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65303780-faa0-4376-8db2-02e3b09867ba service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:16.978167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65303780-faa0-4376-8db2-02e3b09867ba service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:17.035702 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ee63c-07ec-4a43-a446-103e37b1cdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:17.036584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ee63c-07ec-4a43-a446-103e37b1cdcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.036864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ee63c-07ec-4a43-a446-103e37b1cdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.037137 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ee63c-07ec-4a43-a446-103e37b1cdcf admin admin] PATH_INFO: `/v3/projects/3c23c235fa1c4944bd7a337357d0c030` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.038287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ee63c-07ec-4a43-a446-103e37b1cdcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c23c235fa1c4944bd7a337357d0c030)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:17.040210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ee63c-07ec-4a43-a446-103e37b1cdcf admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:17.083742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-560ee63c-07ec-4a43-a446-103e37b1cdcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c23c235fa1c4944bd7a337357d0c030', 'request_id': 'req-560ee63c-07ec-4a43-a446-103e37b1cdcf'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:17.088848 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-560ee63c-07ec-4a43-a446-103e37b1cdcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c23c235fa1c4944bd7a337357d0c030', 'request_id': 'req-560ee63c-07ec-4a43-a446-103e37b1cdcf'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:17.095787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-560ee63c-07ec-4a43-a446-103e37b1cdcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c23c235fa1c4944bd7a337357d0c030', 'request_id': 'req-560ee63c-07ec-4a43-a446-103e37b1cdcf'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:17.105924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-560ee63c-07ec-4a43-a446-103e37b1cdcf admin admin] The token cache is being invalidate because project 3c23c235fa1c4944bd7a337357d0c030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:17.106208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-560ee63c-07ec-4a43-a446-103e37b1cdcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:17.106826 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-560ee63c-07ec-4a43-a446-103e37b1cdcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:17.108348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1210/2385] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:16 2022] DELETE /identity/v3/projects/3c23c235fa1c4944bd7a337357d0c030 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:17.113262 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6b3eb-6de6-44ce-88d8-89f8e11117e3 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.113553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6b3eb-6de6-44ce-88d8-89f8e11117e3 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.113833 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6b3eb-6de6-44ce-88d8-89f8e11117e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.131535 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-24e6b3eb-6de6-44ce-88d8-89f8e11117e3 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:17.132776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-24e6b3eb-6de6-44ce-88d8-89f8e11117e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:17.191380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1176/2386] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:17.199305 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f57d61-54be-4922-9ffc-418d802a75ac None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.199599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f57d61-54be-4922-9ffc-418d802a75ac None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.199881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f57d61-54be-4922-9ffc-418d802a75ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.219671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e4f57d61-54be-4922-9ffc-418d802a75ac None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:17.220910 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e4f57d61-54be-4922-9ffc-418d802a75ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:17.231820 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1211/2387] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:17.233273 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6b3eb-6de6-44ce-88d8-89f8e11117e3 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:17.243031 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6b3eb-6de6-44ce-88d8-89f8e11117e3 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:17.247867 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73266e40-0e78-4fbe-bdb9-f1461c21f7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:17.249161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73266e40-0e78-4fbe-bdb9-f1461c21f7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.249500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73266e40-0e78-4fbe-bdb9-f1461c21f7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.249724 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73266e40-0e78-4fbe-bdb9-f1461c21f7d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.250446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73266e40-0e78-4fbe-bdb9-f1461c21f7d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:33:17.250747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73266e40-0e78-4fbe-bdb9-f1461c21f7d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:17.252469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73266e40-0e78-4fbe-bdb9-f1461c21f7d8 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:17.266703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1177/2388] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:33:17.270713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f57d61-54be-4922-9ffc-418d802a75ac None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:17.280659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f57d61-54be-4922-9ffc-418d802a75ac None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:17.285749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb70899-9559-43a9-8307-d6938c40075f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:17.286418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb70899-9559-43a9-8307-d6938c40075f admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.286698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb70899-9559-43a9-8307-d6938c40075f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.287187 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb70899-9559-43a9-8307-d6938c40075f admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.287918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb70899-9559-43a9-8307-d6938c40075f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:17.290552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb70899-9559-43a9-8307-d6938c40075f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:17.300217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73266e40-0e78-4fbe-bdb9-f1461c21f7d8 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:17.301895 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-73266e40-0e78-4fbe-bdb9-f1461c21f7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:17.315649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73266e40-0e78-4fbe-bdb9-f1461c21f7d8 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:17.321162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebec3ff-0c35-48e1-9e6b-b1cf37e5e929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:17.322213 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebec3ff-0c35-48e1-9e6b-b1cf37e5e929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.322494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebec3ff-0c35-48e1-9e6b-b1cf37e5e929 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.322888 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebec3ff-0c35-48e1-9e6b-b1cf37e5e929 admin admin] PATH_INFO: `/v3/users/2a08c9e49cfc4a55840f17f1124411c9` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.324299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebec3ff-0c35-48e1-9e6b-b1cf37e5e929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a08c9e49cfc4a55840f17f1124411c9)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:17.326199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebec3ff-0c35-48e1-9e6b-b1cf37e5e929 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:17.327232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1212/2389] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:17 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:17.330768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb70899-9559-43a9-8307-d6938c40075f admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:17.387884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5ebec3ff-0c35-48e1-9e6b-b1cf37e5e929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a08c9e49cfc4a55840f17f1124411c9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:17.390132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5ebec3ff-0c35-48e1-9e6b-b1cf37e5e929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a08c9e49cfc4a55840f17f1124411c9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:17.390780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5ebec3ff-0c35-48e1-9e6b-b1cf37e5e929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a08c9e49cfc4a55840f17f1124411c9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:17.400564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5ebec3ff-0c35-48e1-9e6b-b1cf37e5e929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a08c9e49cfc4a55840f17f1124411c9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:17.402944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb70899-9559-43a9-8307-d6938c40075f admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:17.408762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9924a-5f66-42e6-871e-c4e2580cd11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:17.409490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9924a-5f66-42e6-871e-c4e2580cd11a admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.409892 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9924a-5f66-42e6-871e-c4e2580cd11a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.410382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9924a-5f66-42e6-871e-c4e2580cd11a admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.411175 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded9924a-5f66-42e6-871e-c4e2580cd11a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:17.413308 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded9924a-5f66-42e6-871e-c4e2580cd11a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:17.414127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5ebec3ff-0c35-48e1-9e6b-b1cf37e5e929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a08c9e49cfc4a55840f17f1124411c9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:17.437908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1178/2390] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:17 2022] DELETE /identity/v3/users/2a08c9e49cfc4a55840f17f1124411c9 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:17.441444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1213/2391] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:17 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:17.442336 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9924a-5f66-42e6-871e-c4e2580cd11a admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:17.443083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139e997-47a0-4b83-a853-fbc504bfe93d None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.443497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139e997-47a0-4b83-a853-fbc504bfe93d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.443871 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139e997-47a0-4b83-a853-fbc504bfe93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.444253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ded9924a-5f66-42e6-871e-c4e2580cd11a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:17.457670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9924a-5f66-42e6-871e-c4e2580cd11a admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:17.462592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f139e997-47a0-4b83-a853-fbc504bfe93d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:17.464171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f139e997-47a0-4b83-a853-fbc504bfe93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:17.531335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35106ab8-ac01-49fa-abe2-c21d87cae95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:17.532398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35106ab8-ac01-49fa-abe2-c21d87cae95f admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.532851 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35106ab8-ac01-49fa-abe2-c21d87cae95f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.533306 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35106ab8-ac01-49fa-abe2-c21d87cae95f admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.534068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35106ab8-ac01-49fa-abe2-c21d87cae95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:17.535662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35106ab8-ac01-49fa-abe2-c21d87cae95f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:17.543980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1179/2392] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:17.547007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1214/2393] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:17.550662 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139e997-47a0-4b83-a853-fbc504bfe93d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:17.552848 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35106ab8-ac01-49fa-abe2-c21d87cae95f admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:17.554521 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-35106ab8-ac01-49fa-abe2-c21d87cae95f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:17.560039 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139e997-47a0-4b83-a853-fbc504bfe93d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:17.565554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5c3bf-1bbc-45ee-bb5b-85507a9b258d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:17.566209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5c3bf-1bbc-45ee-bb5b-85507a9b258d admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.566488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5c3bf-1bbc-45ee-bb5b-85507a9b258d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.566761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5c3bf-1bbc-45ee-bb5b-85507a9b258d admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.567484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5c3bf-1bbc-45ee-bb5b-85507a9b258d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:17.569080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5c3bf-1bbc-45ee-bb5b-85507a9b258d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:17.578615 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1180/2394] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:33:17.582924 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5c3bf-1bbc-45ee-bb5b-85507a9b258d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:17.592298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5c3bf-1bbc-45ee-bb5b-85507a9b258d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:17.596918 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e76c3-23db-4969-bd93-1475277f13df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:17.597769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e76c3-23db-4969-bd93-1475277f13df admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.598045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e76c3-23db-4969-bd93-1475277f13df admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.598316 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e76c3-23db-4969-bd93-1475277f13df admin admin] PATH_INFO: `/v3/projects/853d056feee44f418d44536348bcfe2b/users/39ae2218b9c64d239fc8487be1708b92/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.610546 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35106ab8-ac01-49fa-abe2-c21d87cae95f admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:17.613276 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e76c3-23db-4969-bd93-1475277f13df admin admin] RBAC: Authorizing `identity:create_grant(project_id=853d056feee44f418d44536348bcfe2b, user_id=39ae2218b9c64d239fc8487be1708b92, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:17.614126 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb110e01-1f0a-4eda-85b9-f653a4d08a33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:17.614764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb110e01-1f0a-4eda-85b9-f653a4d08a33 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.615044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb110e01-1f0a-4eda-85b9-f653a4d08a33 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.615316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb110e01-1f0a-4eda-85b9-f653a4d08a33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.616094 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e76c3-23db-4969-bd93-1475277f13df admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:17.625275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb110e01-1f0a-4eda-85b9-f653a4d08a33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:17.626709 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb110e01-1f0a-4eda-85b9-f653a4d08a33 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:17.629260 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1181/2395] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:17 2022] PUT /identity/v3/projects/853d056feee44f418d44536348bcfe2b/users/39ae2218b9c64d239fc8487be1708b92/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:17.636208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1215/2396] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:33:17.640471 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f09cb5-bfe8-4e5b-a31e-d0358b77b775 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.640758 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f09cb5-bfe8-4e5b-a31e-d0358b77b775 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.641034 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f09cb5-bfe8-4e5b-a31e-d0358b77b775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.666072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-a3f09cb5-bfe8-4e5b-a31e-d0358b77b775 None None] MFA Rules not processed for user `39ae2218b9c64d239fc8487be1708b92`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:17.667331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a3f09cb5-bfe8-4e5b-a31e-d0358b77b775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:17.734909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1182/2397] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:17.747602 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb110e01-1f0a-4eda-85b9-f653a4d08a33 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:17.749490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-fb110e01-1f0a-4eda-85b9-f653a4d08a33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:17.828443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb110e01-1f0a-4eda-85b9-f653a4d08a33 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:17.891548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906629c7-f86d-4c8e-8a50-050fece55de6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:17.892355 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906629c7-f86d-4c8e-8a50-050fece55de6 service nova] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:17.892621 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906629c7-f86d-4c8e-8a50-050fece55de6 service nova] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:17.892915 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906629c7-f86d-4c8e-8a50-050fece55de6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:17.901830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906629c7-f86d-4c8e-8a50-050fece55de6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:17.903669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906629c7-f86d-4c8e-8a50-050fece55de6 service nova] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:17.915031 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f09cb5-bfe8-4e5b-a31e-d0358b77b775 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:17.916532 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a3f09cb5-bfe8-4e5b-a31e-d0358b77b775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:17.924998 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1216/2398] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:17.997289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f09cb5-bfe8-4e5b-a31e-d0358b77b775 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:18.003778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:18.004697 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.004985 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.005263 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] PATH_INFO: `/v3/projects/88cc42b2de4448afbfd3c86b2d474793` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.006460 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88cc42b2de4448afbfd3c86b2d474793)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:18.008390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:18.058819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88cc42b2de4448afbfd3c86b2d474793', 'request_id': 'req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:18.065721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88cc42b2de4448afbfd3c86b2d474793', 'request_id': 'req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:18.071200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88cc42b2de4448afbfd3c86b2d474793', 'request_id': 'req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:18.091133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] The token cache is being invalidate because project 88cc42b2de4448afbfd3c86b2d474793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:18.091133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:18.091133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:18.091133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1183/2399] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:17 2022] DELETE /identity/v3/projects/88cc42b2de4448afbfd3c86b2d474793 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:18.095728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7191272-b60a-4117-b7e6-f77d7c37e247 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.095728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7191272-b60a-4117-b7e6-f77d7c37e247 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.095728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7191272-b60a-4117-b7e6-f77d7c37e247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.112239 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e7191272-b60a-4117-b7e6-f77d7c37e247 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:18.113442 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e7191272-b60a-4117-b7e6-f77d7c37e247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:18.143389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:18.144835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:18.175968 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1217/2400] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:18.184228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2701c-dc88-4b83-a8d7-81e973748036 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.184520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2701c-dc88-4b83-a8d7-81e973748036 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.184797 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2701c-dc88-4b83-a8d7-81e973748036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.204375 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-eda2701c-dc88-4b83-a8d7-81e973748036 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:18.205642 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-eda2701c-dc88-4b83-a8d7-81e973748036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:18.207298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3608a8-cdaf-40fd-8a4c-3b3b55b66dd4 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:18.211928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467073bb-1d13-4626-9c72-84c3119b8f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:18.212711 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467073bb-1d13-4626-9c72-84c3119b8f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.212983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467073bb-1d13-4626-9c72-84c3119b8f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.213255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467073bb-1d13-4626-9c72-84c3119b8f26 admin admin] PATH_INFO: `/v3/users/39ae2218b9c64d239fc8487be1708b92` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.222647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1218/2401] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:18.222799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467073bb-1d13-4626-9c72-84c3119b8f26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39ae2218b9c64d239fc8487be1708b92)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:18.222799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467073bb-1d13-4626-9c72-84c3119b8f26 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:18.223717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2701c-dc88-4b83-a8d7-81e973748036 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:18.232452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2701c-dc88-4b83-a8d7-81e973748036 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:18.238072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19e821-80f4-4567-9c00-8baf43cc411d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:18.238722 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19e821-80f4-4567-9c00-8baf43cc411d admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.239006 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19e821-80f4-4567-9c00-8baf43cc411d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.239276 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19e821-80f4-4567-9c00-8baf43cc411d admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.240024 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19e821-80f4-4567-9c00-8baf43cc411d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:33:18.240324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19e821-80f4-4567-9c00-8baf43cc411d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:18.241894 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19e821-80f4-4567-9c00-8baf43cc411d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:18.254864 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1219/2402] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:33:18.259442 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19e821-80f4-4567-9c00-8baf43cc411d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:18.267840 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-467073bb-1d13-4626-9c72-84c3119b8f26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39ae2218b9c64d239fc8487be1708b92'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:18.268538 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-467073bb-1d13-4626-9c72-84c3119b8f26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39ae2218b9c64d239fc8487be1708b92'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:18.269132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-467073bb-1d13-4626-9c72-84c3119b8f26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39ae2218b9c64d239fc8487be1708b92'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:18.269782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19e821-80f4-4567-9c00-8baf43cc411d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:18.275452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f226c4-dafe-4520-8e65-da921932af9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:18.276314 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f226c4-dafe-4520-8e65-da921932af9e admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.276608 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f226c4-dafe-4520-8e65-da921932af9e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.276891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f226c4-dafe-4520-8e65-da921932af9e admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.277203 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-467073bb-1d13-4626-9c72-84c3119b8f26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39ae2218b9c64d239fc8487be1708b92'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:18.277578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f226c4-dafe-4520-8e65-da921932af9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:18.280615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f226c4-dafe-4520-8e65-da921932af9e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:18.284960 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-467073bb-1d13-4626-9c72-84c3119b8f26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39ae2218b9c64d239fc8487be1708b92'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:18.302718 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1184/2403] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:18 2022] DELETE /identity/v3/users/39ae2218b9c64d239fc8487be1708b92 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:18.307084 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a75e6b-3561-4170-ae2c-08b7f24e04d0 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.307428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a75e6b-3561-4170-ae2c-08b7f24e04d0 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.307716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a75e6b-3561-4170-ae2c-08b7f24e04d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.311777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1220/2404] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:18 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:18.313117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f226c4-dafe-4520-8e65-da921932af9e admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:18.314704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-25f226c4-dafe-4520-8e65-da921932af9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:18.328143 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-58a75e6b-3561-4170-ae2c-08b7f24e04d0 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:18.329328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-58a75e6b-3561-4170-ae2c-08b7f24e04d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:18.377185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f226c4-dafe-4520-8e65-da921932af9e admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:18.381889 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42619ffa-e88d-4ec7-adee-145a72f15a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:18.382580 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619ffa-e88d-4ec7-adee-145a72f15a7a admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.382863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619ffa-e88d-4ec7-adee-145a72f15a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.383174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619ffa-e88d-4ec7-adee-145a72f15a7a admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.383831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42619ffa-e88d-4ec7-adee-145a72f15a7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:18.385423 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1185/2405] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:18.385773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42619ffa-e88d-4ec7-adee-145a72f15a7a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:18.395993 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a75e6b-3561-4170-ae2c-08b7f24e04d0 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:18.397559 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-58a75e6b-3561-4170-ae2c-08b7f24e04d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:18.405858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1221/2406] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:18 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:18.409669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42619ffa-e88d-4ec7-adee-145a72f15a7a admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:18.419102 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42619ffa-e88d-4ec7-adee-145a72f15a7a admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:18.424808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90475bbd-1c39-40fa-8c8d-bf2e004520be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:18.425472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90475bbd-1c39-40fa-8c8d-bf2e004520be admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.425857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90475bbd-1c39-40fa-8c8d-bf2e004520be admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.426138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90475bbd-1c39-40fa-8c8d-bf2e004520be admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.426869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90475bbd-1c39-40fa-8c8d-bf2e004520be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:18.428500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90475bbd-1c39-40fa-8c8d-bf2e004520be admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:18.438639 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1222/2407] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:33:18.442388 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90475bbd-1c39-40fa-8c8d-bf2e004520be admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:18.452503 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a75e6b-3561-4170-ae2c-08b7f24e04d0 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:18.455051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90475bbd-1c39-40fa-8c8d-bf2e004520be admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:18.456352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f7fd4-67bd-4ead-9c81-38d20672150a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:18.457068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f7fd4-67bd-4ead-9c81-38d20672150a service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.457446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f7fd4-67bd-4ead-9c81-38d20672150a service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.457739 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f7fd4-67bd-4ead-9c81-38d20672150a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.459862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881ee66-d4f5-4d78-bbe1-698cd8dd2ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:18.460592 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881ee66-d4f5-4d78-bbe1-698cd8dd2ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.460846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881ee66-d4f5-4d78-bbe1-698cd8dd2ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.461142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881ee66-d4f5-4d78-bbe1-698cd8dd2ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.461857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881ee66-d4f5-4d78-bbe1-698cd8dd2ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:18.463462 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881ee66-d4f5-4d78-bbe1-698cd8dd2ba7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:18.468524 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f7fd4-67bd-4ead-9c81-38d20672150a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:18.470017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f7fd4-67bd-4ead-9c81-38d20672150a service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:18.476493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1223/2408] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:33:18.480155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1186/2409] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:18.480385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881ee66-d4f5-4d78-bbe1-698cd8dd2ba7 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:18.490930 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881ee66-d4f5-4d78-bbe1-698cd8dd2ba7 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:18.495879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cdc3f-c0e7-4005-8fa4-ffd70c65ad3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:18.496798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cdc3f-c0e7-4005-8fa4-ffd70c65ad3a admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.497082 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cdc3f-c0e7-4005-8fa4-ffd70c65ad3a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.497364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cdc3f-c0e7-4005-8fa4-ffd70c65ad3a admin admin] PATH_INFO: `/v3/projects/36e10488543e47f0b786d528ba7e9c2d/users/be37fa6473054ef0bf699c7c1a3d4b59/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.513500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1cdc3f-c0e7-4005-8fa4-ffd70c65ad3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e10488543e47f0b786d528ba7e9c2d, user_id=be37fa6473054ef0bf699c7c1a3d4b59, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:18.516852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1cdc3f-c0e7-4005-8fa4-ffd70c65ad3a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:18.531301 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1224/2410] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:18 2022] PUT /identity/v3/projects/36e10488543e47f0b786d528ba7e9c2d/users/be37fa6473054ef0bf699c7c1a3d4b59/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:18.541734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd65d62-c10e-42d2-b915-7301dec69fa7 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.542018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd65d62-c10e-42d2-b915-7301dec69fa7 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.542290 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd65d62-c10e-42d2-b915-7301dec69fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.573234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-2fd65d62-c10e-42d2-b915-7301dec69fa7 None None] MFA Rules not processed for user `be37fa6473054ef0bf699c7c1a3d4b59`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:18.574482 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-2fd65d62-c10e-42d2-b915-7301dec69fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:18.642213 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1187/2411] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:18.653585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cdc3f-c0e7-4005-8fa4-ffd70c65ad3a admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:18.655395 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2d1cdc3f-c0e7-4005-8fa4-ffd70c65ad3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:18.712141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cdc3f-c0e7-4005-8fa4-ffd70c65ad3a admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:18.716455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae3656-b7ac-46b2-9852-2fd6d83b735b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:18.717202 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae3656-b7ac-46b2-9852-2fd6d83b735b service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.717759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae3656-b7ac-46b2-9852-2fd6d83b735b service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.718166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae3656-b7ac-46b2-9852-2fd6d83b735b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.726878 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae3656-b7ac-46b2-9852-2fd6d83b735b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:18.728465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae3656-b7ac-46b2-9852-2fd6d83b735b service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:18.737628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1225/2412] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:33:18.890259 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd65d62-c10e-42d2-b915-7301dec69fa7 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:18.894081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-2fd65d62-c10e-42d2-b915-7301dec69fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:18.972986 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd65d62-c10e-42d2-b915-7301dec69fa7 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:18.978784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:18.980285 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:18.980644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:18.980989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4 admin admin] PATH_INFO: `/v3/projects/853d056feee44f418d44536348bcfe2b` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:18.982899 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=853d056feee44f418d44536348bcfe2b)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:18.985099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:19.034658 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '853d056feee44f418d44536348bcfe2b', 'request_id': 'req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:19.041242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '853d056feee44f418d44536348bcfe2b', 'request_id': 'req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:19.048153 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '853d056feee44f418d44536348bcfe2b', 'request_id': 'req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:19.065057 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4 admin admin] The token cache is being invalidate because project 853d056feee44f418d44536348bcfe2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:19.066017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:19.066734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-f54e8786-6b5f-4e5a-a040-6bee4e0892a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:19.068512 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1188/2413] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:18 2022] DELETE /identity/v3/projects/853d056feee44f418d44536348bcfe2b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:19.073196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb38ee-d291-46e1-bdfd-8717870972cd None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.073483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb38ee-d291-46e1-bdfd-8717870972cd None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.073822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb38ee-d291-46e1-bdfd-8717870972cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.096071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-5cdb38ee-d291-46e1-bdfd-8717870972cd None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:19.098004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5cdb38ee-d291-46e1-bdfd-8717870972cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:19.171430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1226/2414] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:19.181052 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5ecb-395c-4eed-bbb8-35bd9a390980 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.181546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5ecb-395c-4eed-bbb8-35bd9a390980 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.181958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5ecb-395c-4eed-bbb8-35bd9a390980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.203527 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-1a6f5ecb-395c-4eed-bbb8-35bd9a390980 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:19.204758 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1a6f5ecb-395c-4eed-bbb8-35bd9a390980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:19.212664 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1189/2415] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:19.217258 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb38ee-d291-46e1-bdfd-8717870972cd None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:19.226816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb38ee-d291-46e1-bdfd-8717870972cd None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:19.231373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927a0a7-156c-4966-a4e2-7c84fe730ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:19.232295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927a0a7-156c-4966-a4e2-7c84fe730ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.233090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927a0a7-156c-4966-a4e2-7c84fe730ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.233465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927a0a7-156c-4966-a4e2-7c84fe730ba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.234243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927a0a7-156c-4966-a4e2-7c84fe730ba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:33:19.234545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927a0a7-156c-4966-a4e2-7c84fe730ba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:19.236163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927a0a7-156c-4966-a4e2-7c84fe730ba5 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:19.247793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1227/2416] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:33:19.252742 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f5ecb-395c-4eed-bbb8-35bd9a390980 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:19.261702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f5ecb-395c-4eed-bbb8-35bd9a390980 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:19.266422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cb1a8-4f2c-4961-93aa-807c8e4e43af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:19.267070 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cb1a8-4f2c-4961-93aa-807c8e4e43af admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.267355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cb1a8-4f2c-4961-93aa-807c8e4e43af admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.267630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cb1a8-4f2c-4961-93aa-807c8e4e43af admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.268345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cb1a8-4f2c-4961-93aa-807c8e4e43af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:19.270219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cb1a8-4f2c-4961-93aa-807c8e4e43af admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:19.298755 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1190/2417] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:19 2022] POST /identity/v3/projects => generated 369 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:19.299812 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927a0a7-156c-4966-a4e2-7c84fe730ba5 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:19.309711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927a0a7-156c-4966-a4e2-7c84fe730ba5 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:19.365263 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fae5fe-bbac-4671-99b4-418f51ee2b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:19.365927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fae5fe-bbac-4671-99b4-418f51ee2b77 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.366211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fae5fe-bbac-4671-99b4-418f51ee2b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.366486 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fae5fe-bbac-4671-99b4-418f51ee2b77 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.367175 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fae5fe-bbac-4671-99b4-418f51ee2b77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:19.369151 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fae5fe-bbac-4671-99b4-418f51ee2b77 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:19.388873 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1228/2418] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:19 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:19.394304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cb1a8-4f2c-4961-93aa-807c8e4e43af admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:19.404971 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cb1a8-4f2c-4961-93aa-807c8e4e43af admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:19.409853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3f2c-5592-4e2a-952d-394aedd23f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:19.410504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3f2c-5592-4e2a-952d-394aedd23f7a admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.410813 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3f2c-5592-4e2a-952d-394aedd23f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.411087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3f2c-5592-4e2a-952d-394aedd23f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.411802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa3f2c-5592-4e2a-952d-394aedd23f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:19.413394 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa3f2c-5592-4e2a-952d-394aedd23f7a admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:19.424070 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1191/2419] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:19.427922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fae5fe-bbac-4671-99b4-418f51ee2b77 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:19.438140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fae5fe-bbac-4671-99b4-418f51ee2b77 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:19.442643 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bd8a7-80bf-45a0-9cde-3e3945284130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:19.443292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bd8a7-80bf-45a0-9cde-3e3945284130 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.443569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bd8a7-80bf-45a0-9cde-3e3945284130 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.443840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bd8a7-80bf-45a0-9cde-3e3945284130 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.444617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bd8a7-80bf-45a0-9cde-3e3945284130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:19.446208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bd8a7-80bf-45a0-9cde-3e3945284130 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:19.455817 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1229/2420] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:19.459860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3f2c-5592-4e2a-952d-394aedd23f7a admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:19.470311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3f2c-5592-4e2a-952d-394aedd23f7a admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:19.474717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b4193-27d2-4a08-a11f-bc7424813572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:19.475575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b4193-27d2-4a08-a11f-bc7424813572 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.475856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b4193-27d2-4a08-a11f-bc7424813572 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.476157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b4193-27d2-4a08-a11f-bc7424813572 admin admin] PATH_INFO: `/v3/projects/5fdaeb3fa66449ed962ebc67827a637d/users/7ee0522929fc4d7e89dcdea18aaa49a6/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.490455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b4193-27d2-4a08-a11f-bc7424813572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fdaeb3fa66449ed962ebc67827a637d, user_id=7ee0522929fc4d7e89dcdea18aaa49a6, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:19.493120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b4193-27d2-4a08-a11f-bc7424813572 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:19.505410 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1192/2421] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:19 2022] PUT /identity/v3/projects/5fdaeb3fa66449ed962ebc67827a637d/users/7ee0522929fc4d7e89dcdea18aaa49a6/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:19.515938 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab85cd-3b4a-4f6f-9dd4-db31b8315484 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.516240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab85cd-3b4a-4f6f-9dd4-db31b8315484 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.516554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab85cd-3b4a-4f6f-9dd4-db31b8315484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.542804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-59ab85cd-3b4a-4f6f-9dd4-db31b8315484 None None] MFA Rules not processed for user `7ee0522929fc4d7e89dcdea18aaa49a6`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:19.544034 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-59ab85cd-3b4a-4f6f-9dd4-db31b8315484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:19.604539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1230/2422] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:19 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:19.609558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b4193-27d2-4a08-a11f-bc7424813572 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:19.668845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b4193-27d2-4a08-a11f-bc7424813572 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:19.687683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa1ba5-1509-44c6-9736-56da3cf2dcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:19.687683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa1ba5-1509-44c6-9736-56da3cf2dcc9 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.687683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa1ba5-1509-44c6-9736-56da3cf2dcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.687683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa1ba5-1509-44c6-9736-56da3cf2dcc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.688124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa1ba5-1509-44c6-9736-56da3cf2dcc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:19.688124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa1ba5-1509-44c6-9736-56da3cf2dcc9 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:19.709435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1193/2423] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:19 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:19.713605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab85cd-3b4a-4f6f-9dd4-db31b8315484 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:19.725237 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab85cd-3b4a-4f6f-9dd4-db31b8315484 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:19.778731 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3cca1-9e1f-4fd4-a855-ad17a34e4d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:19.780479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3cca1-9e1f-4fd4-a855-ad17a34e4d46 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.780479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3cca1-9e1f-4fd4-a855-ad17a34e4d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.780795 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3cca1-9e1f-4fd4-a855-ad17a34e4d46 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.781320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3cca1-9e1f-4fd4-a855-ad17a34e4d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:19.783212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3cca1-9e1f-4fd4-a855-ad17a34e4d46 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:19.801612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1231/2424] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:19 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:19.805779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa1ba5-1509-44c6-9736-56da3cf2dcc9 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:19.816185 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa1ba5-1509-44c6-9736-56da3cf2dcc9 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:19.820869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b85a5-8945-4d03-830e-d6fb27c06733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:19.821529 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b85a5-8945-4d03-830e-d6fb27c06733 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.821808 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b85a5-8945-4d03-830e-d6fb27c06733 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.822080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b85a5-8945-4d03-830e-d6fb27c06733 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.822801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b85a5-8945-4d03-830e-d6fb27c06733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:19.824420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b85a5-8945-4d03-830e-d6fb27c06733 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:19.835480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1194/2425] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:19.840019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3cca1-9e1f-4fd4-a855-ad17a34e4d46 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:19.850446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3cca1-9e1f-4fd4-a855-ad17a34e4d46 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:19.855031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81506a1-0576-4d16-a520-02471ae1f8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:19.855843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81506a1-0576-4d16-a520-02471ae1f8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.856149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81506a1-0576-4d16-a520-02471ae1f8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.856421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81506a1-0576-4d16-a520-02471ae1f8a5 admin admin] PATH_INFO: `/v3/projects/82b009d9f0f049d2ba357a2b2b5df1d0/users/5741da0cba4040eab09d0cc91fd69c4a/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.870669 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81506a1-0576-4d16-a520-02471ae1f8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82b009d9f0f049d2ba357a2b2b5df1d0, user_id=5741da0cba4040eab09d0cc91fd69c4a, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:19.873412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81506a1-0576-4d16-a520-02471ae1f8a5 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:19.886837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1232/2426] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:19 2022] PUT /identity/v3/projects/82b009d9f0f049d2ba357a2b2b5df1d0/users/5741da0cba4040eab09d0cc91fd69c4a/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:19.891414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b85a5-8945-4d03-830e-d6fb27c06733 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:19.900998 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b85a5-8945-4d03-830e-d6fb27c06733 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:19.957734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1679cf-1646-4d7d-988e-edf4ea50ab54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:19.958384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1679cf-1646-4d7d-988e-edf4ea50ab54 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.958671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1679cf-1646-4d7d-988e-edf4ea50ab54 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.958949 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1679cf-1646-4d7d-988e-edf4ea50ab54 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.959670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1679cf-1646-4d7d-988e-edf4ea50ab54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:19.961300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1679cf-1646-4d7d-988e-edf4ea50ab54 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:19.970911 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1195/2427] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:19.975013 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81506a1-0576-4d16-a520-02471ae1f8a5 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:19.985590 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81506a1-0576-4d16-a520-02471ae1f8a5 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:19.990173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6ecc6-3cfa-4073-abfc-6acac20cb8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:19.991051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6ecc6-3cfa-4073-abfc-6acac20cb8ae admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:19.991336 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6ecc6-3cfa-4073-abfc-6acac20cb8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:19.991614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6ecc6-3cfa-4073-abfc-6acac20cb8ae admin admin] PATH_INFO: `/v3/projects/82b009d9f0f049d2ba357a2b2b5df1d0/users/5741da0cba4040eab09d0cc91fd69c4a/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:19.993624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6ecc6-3cfa-4073-abfc-6acac20cb8ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=82b009d9f0f049d2ba357a2b2b5df1d0, user_id=5741da0cba4040eab09d0cc91fd69c4a, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:19.996295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d6ecc6-3cfa-4073-abfc-6acac20cb8ae admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:20.007104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1233/2428] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:19 2022] PUT /identity/v3/projects/82b009d9f0f049d2ba357a2b2b5df1d0/users/5741da0cba4040eab09d0cc91fd69c4a/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:20.019105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d440-eee7-4407-8065-c2014c771975 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:20.019397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d440-eee7-4407-8065-c2014c771975 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:20.019674 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d440-eee7-4407-8065-c2014c771975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:20.046251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-2376d440-eee7-4407-8065-c2014c771975 None None] MFA Rules not processed for user `5741da0cba4040eab09d0cc91fd69c4a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:20.047433 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-2376d440-eee7-4407-8065-c2014c771975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:20.113040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1196/2429] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:20.124717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6ecc6-3cfa-4073-abfc-6acac20cb8ae admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:20.126484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e8d6ecc6-3cfa-4073-abfc-6acac20cb8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:20.184458 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d6ecc6-3cfa-4073-abfc-6acac20cb8ae admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:20.188354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f9be3-cca3-4119-98d1-6a6191ef0899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:20.189013 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f9be3-cca3-4119-98d1-6a6191ef0899 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:20.189295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f9be3-cca3-4119-98d1-6a6191ef0899 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:20.189571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f9be3-cca3-4119-98d1-6a6191ef0899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:20.198813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f9be3-cca3-4119-98d1-6a6191ef0899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:20.200262 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f9be3-cca3-4119-98d1-6a6191ef0899 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:20.211405 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1234/2430] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:20 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:20.609167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376d440-eee7-4407-8065-c2014c771975 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:20.671111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376d440-eee7-4407-8065-c2014c771975 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:20.675554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:20.676459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:20.676741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:20.677017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] PATH_INFO: `/v3/users/7ee0522929fc4d7e89dcdea18aaa49a6` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:20.678196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ee0522929fc4d7e89dcdea18aaa49a6)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:20.680136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:20.726143 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ee0522929fc4d7e89dcdea18aaa49a6'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:20.726790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ee0522929fc4d7e89dcdea18aaa49a6'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:20.727396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ee0522929fc4d7e89dcdea18aaa49a6'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:20.735853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ee0522929fc4d7e89dcdea18aaa49a6'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:20.745747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ee0522929fc4d7e89dcdea18aaa49a6'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:20.765871 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1197/2431] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:20 2022] DELETE /identity/v3/users/7ee0522929fc4d7e89dcdea18aaa49a6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:20.770380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcedb96-c2ed-46dd-93bb-495937d3e922 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:20.770671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcedb96-c2ed-46dd-93bb-495937d3e922 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:20.770949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcedb96-c2ed-46dd-93bb-495937d3e922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:20.789194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2fcedb96-c2ed-46dd-93bb-495937d3e922 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:20.790387 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2fcedb96-c2ed-46dd-93bb-495937d3e922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:20.847703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1235/2432] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:20.857349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:20.860029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:20.917129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905de203-47b0-41e0-ab08-4a9abf22e1e0 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:20.921771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f277fae-3ea8-4796-85fc-e3448634670a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:20.922367 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f277fae-3ea8-4796-85fc-e3448634670a service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:20.922681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f277fae-3ea8-4796-85fc-e3448634670a service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:20.922965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f277fae-3ea8-4796-85fc-e3448634670a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:20.931859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f277fae-3ea8-4796-85fc-e3448634670a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:20.933309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f277fae-3ea8-4796-85fc-e3448634670a service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:20.942874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1198/2433] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:21.203724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcedb96-c2ed-46dd-93bb-495937d3e922 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:21.205215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2fcedb96-c2ed-46dd-93bb-495937d3e922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:21.219802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcedb96-c2ed-46dd-93bb-495937d3e922 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:21.225515 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:21.226495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:21.226888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:21.227291 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] PATH_INFO: `/v3/projects/5fdaeb3fa66449ed962ebc67827a637d` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:21.228602 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fdaeb3fa66449ed962ebc67827a637d)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:21.230489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:21.275161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fdaeb3fa66449ed962ebc67827a637d', 'request_id': 'req-6404ded3-0904-445f-96f1-e6baaaf2ce7d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.280532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fdaeb3fa66449ed962ebc67827a637d', 'request_id': 'req-6404ded3-0904-445f-96f1-e6baaaf2ce7d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.287328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fdaeb3fa66449ed962ebc67827a637d', 'request_id': 'req-6404ded3-0904-445f-96f1-e6baaaf2ce7d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.298599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] The token cache is being invalidate because project 5fdaeb3fa66449ed962ebc67827a637d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:21.298887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.299540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.301180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1236/2434] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:21 2022] DELETE /identity/v3/projects/5fdaeb3fa66449ed962ebc67827a637d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:21.310643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f277fae-3ea8-4796-85fc-e3448634670a service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:21.320743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f277fae-3ea8-4796-85fc-e3448634670a service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:21.376957 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c817f-0c29-4b05-a09b-b68043c207cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:21.377817 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c817f-0c29-4b05-a09b-b68043c207cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:21.378098 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c817f-0c29-4b05-a09b-b68043c207cb admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:21.378367 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c817f-0c29-4b05-a09b-b68043c207cb admin admin] PATH_INFO: `/v3/users/5741da0cba4040eab09d0cc91fd69c4a` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:21.379523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c817f-0c29-4b05-a09b-b68043c207cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5741da0cba4040eab09d0cc91fd69c4a)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:21.381455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c817f-0c29-4b05-a09b-b68043c207cb admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:21.424389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a04c817f-0c29-4b05-a09b-b68043c207cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5741da0cba4040eab09d0cc91fd69c4a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.425039 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a04c817f-0c29-4b05-a09b-b68043c207cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5741da0cba4040eab09d0cc91fd69c4a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.425634 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a04c817f-0c29-4b05-a09b-b68043c207cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5741da0cba4040eab09d0cc91fd69c4a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.432204 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a04c817f-0c29-4b05-a09b-b68043c207cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5741da0cba4040eab09d0cc91fd69c4a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.439417 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-a04c817f-0c29-4b05-a09b-b68043c207cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5741da0cba4040eab09d0cc91fd69c4a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.455427 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1199/2435] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:21 2022] DELETE /identity/v3/users/5741da0cba4040eab09d0cc91fd69c4a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:21.739007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:21.740513 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:21.758263 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404ded3-0904-445f-96f1-e6baaaf2ce7d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:21.823357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:21.824385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:21.824734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:21.825185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae admin admin] PATH_INFO: `/v3/projects/82b009d9f0f049d2ba357a2b2b5df1d0` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:21.826408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=82b009d9f0f049d2ba357a2b2b5df1d0)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:21.828555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:21.895123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82b009d9f0f049d2ba357a2b2b5df1d0', 'request_id': 'req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.900599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82b009d9f0f049d2ba357a2b2b5df1d0', 'request_id': 'req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.916463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82b009d9f0f049d2ba357a2b2b5df1d0', 'request_id': 'req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.927802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae admin admin] The token cache is being invalidate because project 82b009d9f0f049d2ba357a2b2b5df1d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:21.928181 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.928814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-9866c294-0166-4cf2-b8b3-6a0d2d6ea6ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:21.930353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1237/2436] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:21 2022] DELETE /identity/v3/projects/82b009d9f0f049d2ba357a2b2b5df1d0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:21.935128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc45c7e-6821-46bd-b291-cce7409082f0 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:21.935677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc45c7e-6821-46bd-b291-cce7409082f0 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:21.935796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc45c7e-6821-46bd-b291-cce7409082f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:21.953364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-0fc45c7e-6821-46bd-b291-cce7409082f0 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:21.954551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0fc45c7e-6821-46bd-b291-cce7409082f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:22.011068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1200/2437] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:22.019388 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01633216-da85-4cea-aa7e-30d4a63043f2 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.019677 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01633216-da85-4cea-aa7e-30d4a63043f2 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.019982 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01633216-da85-4cea-aa7e-30d4a63043f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.038670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-01633216-da85-4cea-aa7e-30d4a63043f2 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:22.039854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-01633216-da85-4cea-aa7e-30d4a63043f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:22.047831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1238/2438] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:22.051931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc45c7e-6821-46bd-b291-cce7409082f0 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.061347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc45c7e-6821-46bd-b291-cce7409082f0 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:22.065811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3eacb-1bc0-488f-b85f-f223252be497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:22.066453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3eacb-1bc0-488f-b85f-f223252be497 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.066734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3eacb-1bc0-488f-b85f-f223252be497 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.067008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3eacb-1bc0-488f-b85f-f223252be497 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.067723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3eacb-1bc0-488f-b85f-f223252be497 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:33:22.068060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3eacb-1bc0-488f-b85f-f223252be497 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:22.069630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3eacb-1bc0-488f-b85f-f223252be497 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:22.082028 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1201/2439] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:33:22.086704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01633216-da85-4cea-aa7e-30d4a63043f2 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.097219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01633216-da85-4cea-aa7e-30d4a63043f2 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:22.101843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbba65-8ff9-4c95-8c16-5894e53432c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:22.102490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbba65-8ff9-4c95-8c16-5894e53432c0 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.102775 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbba65-8ff9-4c95-8c16-5894e53432c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.103054 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbba65-8ff9-4c95-8c16-5894e53432c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.103736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bbba65-8ff9-4c95-8c16-5894e53432c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:22.105656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bbba65-8ff9-4c95-8c16-5894e53432c0 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:22.130404 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1239/2440] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:22 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:22.134604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3eacb-1bc0-488f-b85f-f223252be497 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.144991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3eacb-1bc0-488f-b85f-f223252be497 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:22.199154 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069aa65c-6aff-4bfa-bfcc-b5837cc5c30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:22.199839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069aa65c-6aff-4bfa-bfcc-b5837cc5c30b admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.200147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069aa65c-6aff-4bfa-bfcc-b5837cc5c30b admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.200422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069aa65c-6aff-4bfa-bfcc-b5837cc5c30b admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.201105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069aa65c-6aff-4bfa-bfcc-b5837cc5c30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:22.203981 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069aa65c-6aff-4bfa-bfcc-b5837cc5c30b admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:22.223442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1202/2441] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:22 2022] POST /identity/v3/users => generated 282 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:22.227616 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbba65-8ff9-4c95-8c16-5894e53432c0 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.238052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbba65-8ff9-4c95-8c16-5894e53432c0 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:22.242948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9579d-8c80-4963-a711-56df703d8bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:22.243640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9579d-8c80-4963-a711-56df703d8bce admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.243958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9579d-8c80-4963-a711-56df703d8bce admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.244241 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9579d-8c80-4963-a711-56df703d8bce admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.244993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9579d-8c80-4963-a711-56df703d8bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:22.246603 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9579d-8c80-4963-a711-56df703d8bce admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:22.257350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1240/2442] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:22.261443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069aa65c-6aff-4bfa-bfcc-b5837cc5c30b admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.271722 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069aa65c-6aff-4bfa-bfcc-b5837cc5c30b admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:22.276234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d990a8e-043f-4b68-a85f-7ea2fc55f3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:22.276877 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d990a8e-043f-4b68-a85f-7ea2fc55f3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.277157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d990a8e-043f-4b68-a85f-7ea2fc55f3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.277431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d990a8e-043f-4b68-a85f-7ea2fc55f3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.278142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d990a8e-043f-4b68-a85f-7ea2fc55f3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:22.279917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d990a8e-043f-4b68-a85f-7ea2fc55f3a1 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:22.290079 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1203/2443] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:22.294230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9579d-8c80-4963-a711-56df703d8bce admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.304222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9579d-8c80-4963-a711-56df703d8bce admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:22.308926 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb92735-f8b1-4bb4-b3f9-0a9de94e6bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:22.309808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb92735-f8b1-4bb4-b3f9-0a9de94e6bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.310094 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb92735-f8b1-4bb4-b3f9-0a9de94e6bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.310372 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb92735-f8b1-4bb4-b3f9-0a9de94e6bb1 admin admin] PATH_INFO: `/v3/projects/a7ef1bc254f649899dd2e87721844dc9/users/e08e1c69427941609d626c1185d809a5/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.324775 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb92735-f8b1-4bb4-b3f9-0a9de94e6bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7ef1bc254f649899dd2e87721844dc9, user_id=e08e1c69427941609d626c1185d809a5, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:22.328402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb92735-f8b1-4bb4-b3f9-0a9de94e6bb1 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:22.341078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1241/2444] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:22 2022] PUT /identity/v3/projects/a7ef1bc254f649899dd2e87721844dc9/users/e08e1c69427941609d626c1185d809a5/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:22.351234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c3db7-36df-483e-b502-26df5e72b936 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.351529 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c3db7-36df-483e-b502-26df5e72b936 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.351811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c3db7-36df-483e-b502-26df5e72b936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.377440 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-085c3db7-36df-483e-b502-26df5e72b936 None None] MFA Rules not processed for user `e08e1c69427941609d626c1185d809a5`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:22.378668 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-085c3db7-36df-483e-b502-26df5e72b936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:22.441369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1204/2445] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:22 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:22.445749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb92735-f8b1-4bb4-b3f9-0a9de94e6bb1 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.504517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb92735-f8b1-4bb4-b3f9-0a9de94e6bb1 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:22.509214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5f83f-2707-47a8-9b5c-bcab04b43033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:22.509863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5f83f-2707-47a8-9b5c-bcab04b43033 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.510150 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5f83f-2707-47a8-9b5c-bcab04b43033 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.510427 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5f83f-2707-47a8-9b5c-bcab04b43033 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.511121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5f83f-2707-47a8-9b5c-bcab04b43033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:22.513173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5f83f-2707-47a8-9b5c-bcab04b43033 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:22.538437 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1242/2446] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:22 2022] POST /identity/v3/projects => generated 365 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:22.542452 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c3db7-36df-483e-b502-26df5e72b936 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.552773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c3db7-36df-483e-b502-26df5e72b936 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:22.606817 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdb090-082b-4bcb-bbcd-40a5f0ce82d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:22.608173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdb090-082b-4bcb-bbcd-40a5f0ce82d3 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.608582 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdb090-082b-4bcb-bbcd-40a5f0ce82d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.608865 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdb090-082b-4bcb-bbcd-40a5f0ce82d3 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.609564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdb090-082b-4bcb-bbcd-40a5f0ce82d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:22.611491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdb090-082b-4bcb-bbcd-40a5f0ce82d3 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:22.632126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1205/2447] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:22 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:22.636277 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5f83f-2707-47a8-9b5c-bcab04b43033 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.646787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5f83f-2707-47a8-9b5c-bcab04b43033 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:22.651635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce485e-68bd-4324-b129-6d3cf8139449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:22.652331 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce485e-68bd-4324-b129-6d3cf8139449 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.652619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce485e-68bd-4324-b129-6d3cf8139449 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.652896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce485e-68bd-4324-b129-6d3cf8139449 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.653627 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce485e-68bd-4324-b129-6d3cf8139449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:22.655424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce485e-68bd-4324-b129-6d3cf8139449 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:22.665800 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1243/2448] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:22.669810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdb090-082b-4bcb-bbcd-40a5f0ce82d3 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.680228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdb090-082b-4bcb-bbcd-40a5f0ce82d3 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:22.684712 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c603c-501b-4743-a0ed-e8f8dd21a4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:22.685590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c603c-501b-4743-a0ed-e8f8dd21a4cc admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.686603 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c603c-501b-4743-a0ed-e8f8dd21a4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.686880 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c603c-501b-4743-a0ed-e8f8dd21a4cc admin admin] PATH_INFO: `/v3/projects/f292cdf75fde47e8a9c4cee9ab6ce2d2/users/0764833c478b4b63aa192c6793367ca7/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.700810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c603c-501b-4743-a0ed-e8f8dd21a4cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f292cdf75fde47e8a9c4cee9ab6ce2d2, user_id=0764833c478b4b63aa192c6793367ca7, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:22.703515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c603c-501b-4743-a0ed-e8f8dd21a4cc admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:22.716142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1206/2449] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:22 2022] PUT /identity/v3/projects/f292cdf75fde47e8a9c4cee9ab6ce2d2/users/0764833c478b4b63aa192c6793367ca7/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:22.720233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce485e-68bd-4324-b129-6d3cf8139449 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.729990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce485e-68bd-4324-b129-6d3cf8139449 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:22.786789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab97bad-6e79-405a-be3c-b5211f95a55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:22.787422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab97bad-6e79-405a-be3c-b5211f95a55e admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.787725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab97bad-6e79-405a-be3c-b5211f95a55e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.788029 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab97bad-6e79-405a-be3c-b5211f95a55e admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.788770 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab97bad-6e79-405a-be3c-b5211f95a55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:22.790366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab97bad-6e79-405a-be3c-b5211f95a55e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:22.799771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1244/2450] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:22.804277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c603c-501b-4743-a0ed-e8f8dd21a4cc admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.814701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c603c-501b-4743-a0ed-e8f8dd21a4cc admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:22.819247 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4ae0c-b042-4843-8d40-6b86f0b795f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:22.820149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4ae0c-b042-4843-8d40-6b86f0b795f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.820429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4ae0c-b042-4843-8d40-6b86f0b795f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.820703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4ae0c-b042-4843-8d40-6b86f0b795f1 admin admin] PATH_INFO: `/v3/projects/f292cdf75fde47e8a9c4cee9ab6ce2d2/users/0764833c478b4b63aa192c6793367ca7/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.822640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d4ae0c-b042-4843-8d40-6b86f0b795f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f292cdf75fde47e8a9c4cee9ab6ce2d2, user_id=0764833c478b4b63aa192c6793367ca7, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:22.825336 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d4ae0c-b042-4843-8d40-6b86f0b795f1 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:22.837710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1207/2451] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:22 2022] PUT /identity/v3/projects/f292cdf75fde47e8a9c4cee9ab6ce2d2/users/0764833c478b4b63aa192c6793367ca7/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:22.850037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540699bd-e549-49ea-9596-e12d87dc9936 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:22.850324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540699bd-e549-49ea-9596-e12d87dc9936 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:22.850600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540699bd-e549-49ea-9596-e12d87dc9936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:22.877404 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-540699bd-e549-49ea-9596-e12d87dc9936 None None] MFA Rules not processed for user `0764833c478b4b63aa192c6793367ca7`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:22.878484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-540699bd-e549-49ea-9596-e12d87dc9936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:22.948545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1245/2452] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:22.954022 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4ae0c-b042-4843-8d40-6b86f0b795f1 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.956004 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-88d4ae0c-b042-4843-8d40-6b86f0b795f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:22.962125 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540699bd-e549-49ea-9596-e12d87dc9936 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:22.963455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-540699bd-e549-49ea-9596-e12d87dc9936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:23.019086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540699bd-e549-49ea-9596-e12d87dc9936 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:23.022694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f5fa0-76f2-4fbe-b4bc-7840765283eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:23.023336 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f5fa0-76f2-4fbe-b4bc-7840765283eb service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:23.023617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f5fa0-76f2-4fbe-b4bc-7840765283eb service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:23.023910 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f5fa0-76f2-4fbe-b4bc-7840765283eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:23.033332 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f5fa0-76f2-4fbe-b4bc-7840765283eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:23.034802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f5fa0-76f2-4fbe-b4bc-7840765283eb service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:23.042967 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4ae0c-b042-4843-8d40-6b86f0b795f1 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:23.044252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1246/2453] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:22 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:23.049167 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f16997-aacf-479d-969d-1825511012d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf6107735424ff7901614953e7df403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:23.049847 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f16997-aacf-479d-969d-1825511012d2 service nova] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:23.050028 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f16997-aacf-479d-969d-1825511012d2 service nova] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:23.050293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f16997-aacf-479d-969d-1825511012d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:23.052122 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c1f16997-aacf-479d-969d-1825511012d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:23.115388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f16997-aacf-479d-969d-1825511012d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:23.117028 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f16997-aacf-479d-969d-1825511012d2 service nova] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:23.122169 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f5fa0-76f2-4fbe-b4bc-7840765283eb service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:23.133288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1208/2454] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:23.133413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f5fa0-76f2-4fbe-b4bc-7840765283eb service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:23.135469 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d1c5a-07fd-409a-988c-3a4ed25e4184 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:23.136145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d1c5a-07fd-409a-988c-3a4ed25e4184 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:23.136426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d1c5a-07fd-409a-988c-3a4ed25e4184 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:23.136698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d1c5a-07fd-409a-988c-3a4ed25e4184 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:23.199238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d1c5a-07fd-409a-988c-3a4ed25e4184 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:23.200789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d1c5a-07fd-409a-988c-3a4ed25e4184 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:23.210945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1247/2455] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:23 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:33:23.969857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f16997-aacf-479d-969d-1825511012d2 service nova] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:23.971292 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c1f16997-aacf-479d-969d-1825511012d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:24.026059 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f16997-aacf-479d-969d-1825511012d2 service nova] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:24.030335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a45658-de09-4d4e-8e61-1a0cb5f85af5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:24.030987 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a45658-de09-4d4e-8e61-1a0cb5f85af5 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:24.031266 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a45658-de09-4d4e-8e61-1a0cb5f85af5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:24.031542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a45658-de09-4d4e-8e61-1a0cb5f85af5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:24.033442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-26a45658-de09-4d4e-8e61-1a0cb5f85af5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:24.090660 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a45658-de09-4d4e-8e61-1a0cb5f85af5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:24.092602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a45658-de09-4d4e-8e61-1a0cb5f85af5 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:24.101530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1209/2456] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:26.101372 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d1c5a-07fd-409a-988c-3a4ed25e4184 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:26.175364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d1c5a-07fd-409a-988c-3a4ed25e4184 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:26.179885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:26.180816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:26.181232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:26.181705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] PATH_INFO: `/v3/users/e08e1c69427941609d626c1185d809a5` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:26.183020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e08e1c69427941609d626c1185d809a5)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:26.185003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:26.239584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e08e1c69427941609d626c1185d809a5'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:26.240472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e08e1c69427941609d626c1185d809a5'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:26.248608 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e08e1c69427941609d626c1185d809a5'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:26.255768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e08e1c69427941609d626c1185d809a5'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:26.256461 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e08e1c69427941609d626c1185d809a5'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:26.274186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1248/2457] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:26 2022] DELETE /identity/v3/users/e08e1c69427941609d626c1185d809a5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:26.278628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafc70e-9b61-48a7-889e-5111c33a5042 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:26.279100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafc70e-9b61-48a7-889e-5111c33a5042 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:26.279196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafc70e-9b61-48a7-889e-5111c33a5042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:26.296608 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-bfafc70e-9b61-48a7-889e-5111c33a5042 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:26.297801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-bfafc70e-9b61-48a7-889e-5111c33a5042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:26.359348 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1210/2458] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:26.369096 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:26.370761 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:26.430448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c54c12-0361-4a79-8dba-edce174b3b7c admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:26.434152 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da14f5-3431-4548-a7f5-391f21fc069d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:26.434821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da14f5-3431-4548-a7f5-391f21fc069d service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:26.435107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da14f5-3431-4548-a7f5-391f21fc069d service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:26.435386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da14f5-3431-4548-a7f5-391f21fc069d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:26.445568 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da14f5-3431-4548-a7f5-391f21fc069d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:26.447016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da14f5-3431-4548-a7f5-391f21fc069d service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:26.458334 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1249/2459] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:26.734620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafc70e-9b61-48a7-889e-5111c33a5042 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:26.736299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-bfafc70e-9b61-48a7-889e-5111c33a5042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:26.756886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafc70e-9b61-48a7-889e-5111c33a5042 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:26.770644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:26.771525 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:26.771803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:26.772205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] PATH_INFO: `/v3/projects/a7ef1bc254f649899dd2e87721844dc9` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:26.773385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7ef1bc254f649899dd2e87721844dc9)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:26.775333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:26.832473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7ef1bc254f649899dd2e87721844dc9', 'request_id': 'req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:26.838816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7ef1bc254f649899dd2e87721844dc9', 'request_id': 'req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:26.845069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7ef1bc254f649899dd2e87721844dc9', 'request_id': 'req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:26.859366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] The token cache is being invalidate because project a7ef1bc254f649899dd2e87721844dc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:26.859671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:26.860321 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:26.862195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1211/2460] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:26 2022] DELETE /identity/v3/projects/a7ef1bc254f649899dd2e87721844dc9 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:26.866191 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da14f5-3431-4548-a7f5-391f21fc069d service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:26.877304 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da14f5-3431-4548-a7f5-391f21fc069d service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:26.944353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879fd59-db57-4206-bff8-7d4121d92271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:26.945431 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879fd59-db57-4206-bff8-7d4121d92271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:26.945769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879fd59-db57-4206-bff8-7d4121d92271 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:26.946105 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879fd59-db57-4206-bff8-7d4121d92271 admin admin] PATH_INFO: `/v3/users/0764833c478b4b63aa192c6793367ca7` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:26.947392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879fd59-db57-4206-bff8-7d4121d92271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0764833c478b4b63aa192c6793367ca7)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:26.949518 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879fd59-db57-4206-bff8-7d4121d92271 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:26.994851 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c879fd59-db57-4206-bff8-7d4121d92271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0764833c478b4b63aa192c6793367ca7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:26.995491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c879fd59-db57-4206-bff8-7d4121d92271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0764833c478b4b63aa192c6793367ca7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:27.003184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c879fd59-db57-4206-bff8-7d4121d92271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0764833c478b4b63aa192c6793367ca7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:27.009752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c879fd59-db57-4206-bff8-7d4121d92271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0764833c478b4b63aa192c6793367ca7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:27.010730 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c879fd59-db57-4206-bff8-7d4121d92271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0764833c478b4b63aa192c6793367ca7'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:27.029118 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1250/2461] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:26 2022] DELETE /identity/v3/users/0764833c478b4b63aa192c6793367ca7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:27.328689 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:27.330717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:27.346972 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce77d5-2cd7-487e-996a-3da8ae7e8c3b admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:27.407069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:27.408082 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:27.408369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:27.408639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91 admin admin] PATH_INFO: `/v3/projects/f292cdf75fde47e8a9c4cee9ab6ce2d2` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:27.409835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f292cdf75fde47e8a9c4cee9ab6ce2d2)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:27.411717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:27.459267 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f292cdf75fde47e8a9c4cee9ab6ce2d2', 'request_id': 'req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:27.465358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f292cdf75fde47e8a9c4cee9ab6ce2d2', 'request_id': 'req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:27.470828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f292cdf75fde47e8a9c4cee9ab6ce2d2', 'request_id': 'req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:27.484754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91 admin admin] The token cache is being invalidate because project f292cdf75fde47e8a9c4cee9ab6ce2d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:27.485046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:27.485701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-bcc6e22d-902e-4360-81ff-5d73dcd6fc91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:27.487263 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1212/2462] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:27 2022] DELETE /identity/v3/projects/f292cdf75fde47e8a9c4cee9ab6ce2d2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:27.492211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8ebb7-f006-4bdf-88a8-a2f28b498347 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:27.492713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8ebb7-f006-4bdf-88a8-a2f28b498347 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:27.492946 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8ebb7-f006-4bdf-88a8-a2f28b498347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:27.510657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2ea8ebb7-f006-4bdf-88a8-a2f28b498347 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:27.512025 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2ea8ebb7-f006-4bdf-88a8-a2f28b498347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:27.574403 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1251/2463] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:27.582305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b31f8-b910-412c-84af-b08396b1d1a0 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:27.582833 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b31f8-b910-412c-84af-b08396b1d1a0 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:27.583143 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b31f8-b910-412c-84af-b08396b1d1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:27.604682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-888b31f8-b910-412c-84af-b08396b1d1a0 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:27.605887 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-888b31f8-b910-412c-84af-b08396b1d1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:27.615558 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1213/2464] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:27.619765 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8ebb7-f006-4bdf-88a8-a2f28b498347 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:27.631240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8ebb7-f006-4bdf-88a8-a2f28b498347 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:27.637487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725149c2-7a09-4c78-90ae-fa04ad20718e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:27.638286 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725149c2-7a09-4c78-90ae-fa04ad20718e admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:27.638576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725149c2-7a09-4c78-90ae-fa04ad20718e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:27.639694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725149c2-7a09-4c78-90ae-fa04ad20718e admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:27.640493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725149c2-7a09-4c78-90ae-fa04ad20718e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:33:27.640806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725149c2-7a09-4c78-90ae-fa04ad20718e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:27.642434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725149c2-7a09-4c78-90ae-fa04ad20718e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:27.656590 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1252/2465] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:33:27.660807 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b31f8-b910-412c-84af-b08396b1d1a0 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:27.670822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b31f8-b910-412c-84af-b08396b1d1a0 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:27.676517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50302b5c-ff3b-4af5-9dec-267e0a16ee76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:27.677295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50302b5c-ff3b-4af5-9dec-267e0a16ee76 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:27.678004 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50302b5c-ff3b-4af5-9dec-267e0a16ee76 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:27.678593 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50302b5c-ff3b-4af5-9dec-267e0a16ee76 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:27.679497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50302b5c-ff3b-4af5-9dec-267e0a16ee76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:27.681563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50302b5c-ff3b-4af5-9dec-267e0a16ee76 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:27.706823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1214/2466] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:27 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:27.711194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725149c2-7a09-4c78-90ae-fa04ad20718e admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:27.722426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725149c2-7a09-4c78-90ae-fa04ad20718e admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:27.776582 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39ed40-762f-4253-b89a-1d8ea7e6b299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:27.777307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ed40-762f-4253-b89a-1d8ea7e6b299 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:27.777586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ed40-762f-4253-b89a-1d8ea7e6b299 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:27.777860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ed40-762f-4253-b89a-1d8ea7e6b299 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:27.778553 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39ed40-762f-4253-b89a-1d8ea7e6b299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:27.781115 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39ed40-762f-4253-b89a-1d8ea7e6b299 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:27.803260 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1253/2467] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:27 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:27.804765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50302b5c-ff3b-4af5-9dec-267e0a16ee76 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:27.816530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50302b5c-ff3b-4af5-9dec-267e0a16ee76 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:27.821155 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc020f-5cca-48c2-b181-db0e2b8a83fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:27.821859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc020f-5cca-48c2-b181-db0e2b8a83fb admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:27.822133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc020f-5cca-48c2-b181-db0e2b8a83fb admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:27.822639 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc020f-5cca-48c2-b181-db0e2b8a83fb admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:27.823363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc020f-5cca-48c2-b181-db0e2b8a83fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:27.824958 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc020f-5cca-48c2-b181-db0e2b8a83fb admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:27.836350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1215/2468] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:27.840691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39ed40-762f-4253-b89a-1d8ea7e6b299 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:27.850637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39ed40-762f-4253-b89a-1d8ea7e6b299 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:27.855356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa54f2-979f-4df0-bcc4-1147710a30e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:27.856048 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa54f2-979f-4df0-bcc4-1147710a30e2 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:27.856331 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa54f2-979f-4df0-bcc4-1147710a30e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:27.856603 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa54f2-979f-4df0-bcc4-1147710a30e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:27.857928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa54f2-979f-4df0-bcc4-1147710a30e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:27.859733 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa54f2-979f-4df0-bcc4-1147710a30e2 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:27.871764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1254/2469] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:27.876823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc020f-5cca-48c2-b181-db0e2b8a83fb admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:27.886466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc020f-5cca-48c2-b181-db0e2b8a83fb admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:27.891121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9fb39-1545-4be4-90f5-87665281e15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:27.892102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9fb39-1545-4be4-90f5-87665281e15f admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:27.892489 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9fb39-1545-4be4-90f5-87665281e15f admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:27.892666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9fb39-1545-4be4-90f5-87665281e15f admin admin] PATH_INFO: `/v3/projects/339193812dc944b79484248ed4a26814/users/29633d0e0bde43438ccf8582f8892633/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:27.910123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9fb39-1545-4be4-90f5-87665281e15f admin admin] RBAC: Authorizing `identity:create_grant(project_id=339193812dc944b79484248ed4a26814, user_id=29633d0e0bde43438ccf8582f8892633, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:27.913872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9fb39-1545-4be4-90f5-87665281e15f admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:27.929523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1216/2470] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:27 2022] PUT /identity/v3/projects/339193812dc944b79484248ed4a26814/users/29633d0e0bde43438ccf8582f8892633/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:27.942119 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac29f7-d858-42aa-8a4f-a1ac562015a5 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:27.942413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac29f7-d858-42aa-8a4f-a1ac562015a5 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:27.942703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac29f7-d858-42aa-8a4f-a1ac562015a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:27.988632 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e1ac29f7-d858-42aa-8a4f-a1ac562015a5 None None] MFA Rules not processed for user `29633d0e0bde43438ccf8582f8892633`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:27.989837 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e1ac29f7-d858-42aa-8a4f-a1ac562015a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:28.086470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1255/2471] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:28.098506 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9fb39-1545-4be4-90f5-87665281e15f admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:28.104200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-24f9fb39-1545-4be4-90f5-87665281e15f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:28.173493 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9fb39-1545-4be4-90f5-87665281e15f admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:28.177543 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fb983-d005-4b59-88f0-60d75d8436ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:28.178335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fb983-d005-4b59-88f0-60d75d8436ac service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:28.178736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fb983-d005-4b59-88f0-60d75d8436ac service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:28.179083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fb983-d005-4b59-88f0-60d75d8436ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:28.188059 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727fb983-d005-4b59-88f0-60d75d8436ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:28.189493 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727fb983-d005-4b59-88f0-60d75d8436ac service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:28.198757 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1217/2472] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:28.221366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac29f7-d858-42aa-8a4f-a1ac562015a5 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:28.287452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac29f7-d858-42aa-8a4f-a1ac562015a5 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:28.293471 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:28.294225 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:28.294504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:28.294792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] PATH_INFO: `/v3/users/29633d0e0bde43438ccf8582f8892633` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:28.297165 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29633d0e0bde43438ccf8582f8892633)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:28.298975 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:28.354694 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29633d0e0bde43438ccf8582f8892633'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:28.355347 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29633d0e0bde43438ccf8582f8892633'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:28.364320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29633d0e0bde43438ccf8582f8892633'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:28.371458 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29633d0e0bde43438ccf8582f8892633'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:28.372251 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29633d0e0bde43438ccf8582f8892633'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:28.391772 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1256/2473] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:28 2022] DELETE /identity/v3/users/29633d0e0bde43438ccf8582f8892633 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:28.396573 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761b310-c520-4acc-ac82-4d3bd8eaa296 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:28.397100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761b310-c520-4acc-ac82-4d3bd8eaa296 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:28.397204 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761b310-c520-4acc-ac82-4d3bd8eaa296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:28.415830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-1761b310-c520-4acc-ac82-4d3bd8eaa296 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:28.416998 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1761b310-c520-4acc-ac82-4d3bd8eaa296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:28.475169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1218/2474] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:28.484782 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:28.486184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:28.547080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b91af2-8119-4523-9b78-16f9702eaaf7 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:28.550773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7af15-4624-4460-9689-ffac3b2a8681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:28.551422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7af15-4624-4460-9689-ffac3b2a8681 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:28.551708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7af15-4624-4460-9689-ffac3b2a8681 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:28.552018 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7af15-4624-4460-9689-ffac3b2a8681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:28.561612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7af15-4624-4460-9689-ffac3b2a8681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:28.563085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7af15-4624-4460-9689-ffac3b2a8681 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:28.572384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1257/2475] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:28.827351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761b310-c520-4acc-ac82-4d3bd8eaa296 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:28.828785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1761b310-c520-4acc-ac82-4d3bd8eaa296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:28.840560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761b310-c520-4acc-ac82-4d3bd8eaa296 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:28.845416 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee6957-33ca-4df5-bfed-a33e50693af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:28.846305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee6957-33ca-4df5-bfed-a33e50693af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:28.846588 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee6957-33ca-4df5-bfed-a33e50693af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:28.846858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee6957-33ca-4df5-bfed-a33e50693af8 admin admin] PATH_INFO: `/v3/projects/339193812dc944b79484248ed4a26814` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:28.848055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee6957-33ca-4df5-bfed-a33e50693af8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=339193812dc944b79484248ed4a26814)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:28.849938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee6957-33ca-4df5-bfed-a33e50693af8 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:28.896513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4eee6957-33ca-4df5-bfed-a33e50693af8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '339193812dc944b79484248ed4a26814', 'request_id': 'req-4eee6957-33ca-4df5-bfed-a33e50693af8'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:28.902683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4eee6957-33ca-4df5-bfed-a33e50693af8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '339193812dc944b79484248ed4a26814', 'request_id': 'req-4eee6957-33ca-4df5-bfed-a33e50693af8'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:28.909432 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4eee6957-33ca-4df5-bfed-a33e50693af8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '339193812dc944b79484248ed4a26814', 'request_id': 'req-4eee6957-33ca-4df5-bfed-a33e50693af8'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:28.921927 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4eee6957-33ca-4df5-bfed-a33e50693af8 admin admin] The token cache is being invalidate because project 339193812dc944b79484248ed4a26814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:28.922209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4eee6957-33ca-4df5-bfed-a33e50693af8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:28.922836 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4eee6957-33ca-4df5-bfed-a33e50693af8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:28.924454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1219/2476] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:28 2022] DELETE /identity/v3/projects/339193812dc944b79484248ed4a26814 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:28.929430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c783521-87bd-4ae8-8fca-f7f1bafa32e1 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:28.929719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c783521-87bd-4ae8-8fca-f7f1bafa32e1 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:28.929996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c783521-87bd-4ae8-8fca-f7f1bafa32e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:28.947576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-9c783521-87bd-4ae8-8fca-f7f1bafa32e1 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:28.948800 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-9c783521-87bd-4ae8-8fca-f7f1bafa32e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:29.005927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1258/2477] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:29.013587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf57c9-16d1-479f-bf5c-ce70a1a121ea None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.013899 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf57c9-16d1-479f-bf5c-ce70a1a121ea None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.014177 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf57c9-16d1-479f-bf5c-ce70a1a121ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.032314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-14bf57c9-16d1-479f-bf5c-ce70a1a121ea None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:29.033504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-14bf57c9-16d1-479f-bf5c-ce70a1a121ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:29.041322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1220/2478] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:29.046041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c783521-87bd-4ae8-8fca-f7f1bafa32e1 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.055340 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c783521-87bd-4ae8-8fca-f7f1bafa32e1 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.059822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a84b2-6157-4013-a8a6-c9367887e892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.060570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a84b2-6157-4013-a8a6-c9367887e892 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.061036 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a84b2-6157-4013-a8a6-c9367887e892 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.061426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a84b2-6157-4013-a8a6-c9367887e892 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.062150 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a84b2-6157-4013-a8a6-c9367887e892 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:33:29.062459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a84b2-6157-4013-a8a6-c9367887e892 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.064072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a84b2-6157-4013-a8a6-c9367887e892 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.075086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1259/2479] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:33:29.079230 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf57c9-16d1-479f-bf5c-ce70a1a121ea None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.088120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf57c9-16d1-479f-bf5c-ce70a1a121ea None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.093426 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e8a52-d0a8-416c-a8f5-b8e17f44f989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.094056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e8a52-d0a8-416c-a8f5-b8e17f44f989 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.094335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e8a52-d0a8-416c-a8f5-b8e17f44f989 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.094607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e8a52-d0a8-416c-a8f5-b8e17f44f989 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.095289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e8a52-d0a8-416c-a8f5-b8e17f44f989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.097184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e8a52-d0a8-416c-a8f5-b8e17f44f989 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.122649 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1221/2480] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:29 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:29.126986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a84b2-6157-4013-a8a6-c9367887e892 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.137065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e8a52-d0a8-416c-a8f5-b8e17f44f989 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.137401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a84b2-6157-4013-a8a6-c9367887e892 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.138740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-6b9e8a52-d0a8-416c-a8f5-b8e17f44f989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:29.205553 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc5446-4518-40af-9a9a-64f61afb4e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.206220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc5446-4518-40af-9a9a-64f61afb4e0d admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.206511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc5446-4518-40af-9a9a-64f61afb4e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.206795 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc5446-4518-40af-9a9a-64f61afb4e0d admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.207502 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc5446-4518-40af-9a9a-64f61afb4e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.209533 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc5446-4518-40af-9a9a-64f61afb4e0d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.212359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e8a52-d0a8-416c-a8f5-b8e17f44f989 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.217931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9965e5-b3f3-4a79-a9d1-f6ba8be5171b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.218905 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9965e5-b3f3-4a79-a9d1-f6ba8be5171b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.219198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9965e5-b3f3-4a79-a9d1-f6ba8be5171b admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.219476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9965e5-b3f3-4a79-a9d1-f6ba8be5171b admin admin] PATH_INFO: `/v3/users/be37fa6473054ef0bf699c7c1a3d4b59` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.220722 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9965e5-b3f3-4a79-a9d1-f6ba8be5171b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be37fa6473054ef0bf699c7c1a3d4b59)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.222618 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9965e5-b3f3-4a79-a9d1-f6ba8be5171b admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.233949 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1260/2481] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:29 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:29.238986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc5446-4518-40af-9a9a-64f61afb4e0d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.249498 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc5446-4518-40af-9a9a-64f61afb4e0d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.255872 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3d6e6-ba3c-460a-b052-dce989b80588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.257374 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3d6e6-ba3c-460a-b052-dce989b80588 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.257807 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3d6e6-ba3c-460a-b052-dce989b80588 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.258263 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3d6e6-ba3c-460a-b052-dce989b80588 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.259228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3d6e6-ba3c-460a-b052-dce989b80588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.261080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3d6e6-ba3c-460a-b052-dce989b80588 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.274692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1261/2482] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:33:29.278774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3d6e6-ba3c-460a-b052-dce989b80588 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.281002 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-fc9965e5-b3f3-4a79-a9d1-f6ba8be5171b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be37fa6473054ef0bf699c7c1a3d4b59'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:29.281668 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-fc9965e5-b3f3-4a79-a9d1-f6ba8be5171b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be37fa6473054ef0bf699c7c1a3d4b59'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:29.282328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-fc9965e5-b3f3-4a79-a9d1-f6ba8be5171b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be37fa6473054ef0bf699c7c1a3d4b59'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:29.289080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3d6e6-ba3c-460a-b052-dce989b80588 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.291702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-fc9965e5-b3f3-4a79-a9d1-f6ba8be5171b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be37fa6473054ef0bf699c7c1a3d4b59'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:29.294496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ba27f-0cfc-4b8a-afd5-650e74fcb98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.295999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ba27f-0cfc-4b8a-afd5-650e74fcb98c admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.296281 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ba27f-0cfc-4b8a-afd5-650e74fcb98c admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.296570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ba27f-0cfc-4b8a-afd5-650e74fcb98c admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.297351 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ba27f-0cfc-4b8a-afd5-650e74fcb98c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.299111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ba27f-0cfc-4b8a-afd5-650e74fcb98c admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.299240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-fc9965e5-b3f3-4a79-a9d1-f6ba8be5171b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be37fa6473054ef0bf699c7c1a3d4b59'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:29.315471 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1262/2483] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:33:29.318468 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ba27f-0cfc-4b8a-afd5-650e74fcb98c admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.319231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1222/2484] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:29 2022] DELETE /identity/v3/users/be37fa6473054ef0bf699c7c1a3d4b59 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:29.321114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-178ba27f-0cfc-4b8a-afd5-650e74fcb98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:29.325610 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2545768-3a36-4ab5-8376-e4f3d0257e40 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.326910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2545768-3a36-4ab5-8376-e4f3d0257e40 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.326910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2545768-3a36-4ab5-8376-e4f3d0257e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.346656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f2545768-3a36-4ab5-8376-e4f3d0257e40 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:29.351289 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f2545768-3a36-4ab5-8376-e4f3d0257e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:29.361652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ba27f-0cfc-4b8a-afd5-650e74fcb98c admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.438928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1223/2485] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:29.439866 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2545768-3a36-4ab5-8376-e4f3d0257e40 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.441451 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f2545768-3a36-4ab5-8376-e4f3d0257e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:29.460502 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5cdb3-9e72-40f6-8c6f-2fff20e1a379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.461479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5cdb3-9e72-40f6-8c6f-2fff20e1a379 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.461757 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5cdb3-9e72-40f6-8c6f-2fff20e1a379 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.462051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5cdb3-9e72-40f6-8c6f-2fff20e1a379 admin admin] PATH_INFO: `/v3/projects/90a2839af354461eb78797a1ca8291c6/users/ee43672d56b248889eff989eedda263d/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.476917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5cdb3-9e72-40f6-8c6f-2fff20e1a379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a2839af354461eb78797a1ca8291c6, user_id=ee43672d56b248889eff989eedda263d, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.479538 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5cdb3-9e72-40f6-8c6f-2fff20e1a379 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.492855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1263/2486] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:29 2022] PUT /identity/v3/projects/90a2839af354461eb78797a1ca8291c6/users/ee43672d56b248889eff989eedda263d/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:29.496801 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5cdb3-9e72-40f6-8c6f-2fff20e1a379 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.499055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-eea5cdb3-9e72-40f6-8c6f-2fff20e1a379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:29.501507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2545768-3a36-4ab5-8376-e4f3d0257e40 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.549306 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3c922-2062-4ba1-9c57-1de334d71267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.549870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3c922-2062-4ba1-9c57-1de334d71267 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.550147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3c922-2062-4ba1-9c57-1de334d71267 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.550420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3c922-2062-4ba1-9c57-1de334d71267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.552247 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-52a3c922-2062-4ba1-9c57-1de334d71267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:29.554852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5cdb3-9e72-40f6-8c6f-2fff20e1a379 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.558483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe552e-a7ea-4363-b23c-3e5ea53191a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.559153 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe552e-a7ea-4363-b23c-3e5ea53191a5 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.559435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe552e-a7ea-4363-b23c-3e5ea53191a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.559715 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe552e-a7ea-4363-b23c-3e5ea53191a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.616259 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3c922-2062-4ba1-9c57-1de334d71267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.617669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3c922-2062-4ba1-9c57-1de334d71267 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.618853 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe552e-a7ea-4363-b23c-3e5ea53191a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.620324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe552e-a7ea-4363-b23c-3e5ea53191a5 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.628268 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1224/2487] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:29.629547 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a364b-bfb4-4fd0-9e17-5b3cf294ebb2 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.630007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a364b-bfb4-4fd0-9e17-5b3cf294ebb2 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.630518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a364b-bfb4-4fd0-9e17-5b3cf294ebb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.630670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1264/2488] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:29.658545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-934a364b-bfb4-4fd0-9e17-5b3cf294ebb2 None None] MFA Rules not processed for user `ee43672d56b248889eff989eedda263d`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:29.659762 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-934a364b-bfb4-4fd0-9e17-5b3cf294ebb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:29.730196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1225/2489] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:29.735593 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe552e-a7ea-4363-b23c-3e5ea53191a5 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.746070 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe552e-a7ea-4363-b23c-3e5ea53191a5 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.751777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65724a-3076-4acd-89a1-9c02398d0187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.752611 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65724a-3076-4acd-89a1-9c02398d0187 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.752892 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65724a-3076-4acd-89a1-9c02398d0187 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.753170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65724a-3076-4acd-89a1-9c02398d0187 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.759052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65724a-3076-4acd-89a1-9c02398d0187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.759052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65724a-3076-4acd-89a1-9c02398d0187 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.784351 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1265/2490] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:29 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:29.788384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a364b-bfb4-4fd0-9e17-5b3cf294ebb2 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.800797 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a364b-bfb4-4fd0-9e17-5b3cf294ebb2 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.864735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a7eb6-66ba-4a3b-a095-5f53ad70b5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.866091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a7eb6-66ba-4a3b-a095-5f53ad70b5ad admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.866671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a7eb6-66ba-4a3b-a095-5f53ad70b5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.867188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a7eb6-66ba-4a3b-a095-5f53ad70b5ad admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.868595 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a7eb6-66ba-4a3b-a095-5f53ad70b5ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.872410 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a7eb6-66ba-4a3b-a095-5f53ad70b5ad admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.897784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1226/2491] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:29 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:29.901689 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65724a-3076-4acd-89a1-9c02398d0187 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.916966 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65724a-3076-4acd-89a1-9c02398d0187 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.927751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6810c45-efeb-49cc-a90f-5766fb35b499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.927751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6810c45-efeb-49cc-a90f-5766fb35b499 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.928228 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6810c45-efeb-49cc-a90f-5766fb35b499 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.928559 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6810c45-efeb-49cc-a90f-5766fb35b499 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.929754 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6810c45-efeb-49cc-a90f-5766fb35b499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.931651 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6810c45-efeb-49cc-a90f-5766fb35b499 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.943328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a7eb6-66ba-4a3b-a095-5f53ad70b5ad admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.946014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b87a7eb6-66ba-4a3b-a095-5f53ad70b5ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:29.954914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1266/2492] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:33:29.958885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6810c45-efeb-49cc-a90f-5766fb35b499 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:29.961520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a7eb6-66ba-4a3b-a095-5f53ad70b5ad admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.970148 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6810c45-efeb-49cc-a90f-5766fb35b499 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:29.973031 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501539b5-b2ae-4583-ab8e-79508bc24a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.974554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501539b5-b2ae-4583-ab8e-79508bc24a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.974948 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501539b5-b2ae-4583-ab8e-79508bc24a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.975297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501539b5-b2ae-4583-ab8e-79508bc24a82 admin admin] PATH_INFO: `/v3/projects/36e10488543e47f0b786d528ba7e9c2d` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.975406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6944176-761f-4bd0-bdfa-ae250736e40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:29.976112 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6944176-761f-4bd0-bdfa-ae250736e40d admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:29.976395 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6944176-761f-4bd0-bdfa-ae250736e40d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:29.976690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6944176-761f-4bd0-bdfa-ae250736e40d admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:29.976783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501539b5-b2ae-4583-ab8e-79508bc24a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36e10488543e47f0b786d528ba7e9c2d)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.977384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6944176-761f-4bd0-bdfa-ae250736e40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:29.979135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6944176-761f-4bd0-bdfa-ae250736e40d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.980012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501539b5-b2ae-4583-ab8e-79508bc24a82 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:29.990125 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1267/2493] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:33:29.994130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6944176-761f-4bd0-bdfa-ae250736e40d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.006055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6944176-761f-4bd0-bdfa-ae250736e40d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.011134 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dc18b-ca47-45e3-9847-771fcf76b2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.012143 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dc18b-ca47-45e3-9847-771fcf76b2f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.012307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dc18b-ca47-45e3-9847-771fcf76b2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.012586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dc18b-ca47-45e3-9847-771fcf76b2f4 admin admin] PATH_INFO: `/v3/projects/85f660610db34845b21dcebe5721eccb/users/d465a041744f45e4be3d22cffd4de349/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.028248 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66dc18b-ca47-45e3-9847-771fcf76b2f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85f660610db34845b21dcebe5721eccb, user_id=d465a041744f45e4be3d22cffd4de349, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.031417 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66dc18b-ca47-45e3-9847-771fcf76b2f4 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.032822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-501539b5-b2ae-4583-ab8e-79508bc24a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36e10488543e47f0b786d528ba7e9c2d', 'request_id': 'req-501539b5-b2ae-4583-ab8e-79508bc24a82'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:30.039596 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-501539b5-b2ae-4583-ab8e-79508bc24a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36e10488543e47f0b786d528ba7e9c2d', 'request_id': 'req-501539b5-b2ae-4583-ab8e-79508bc24a82'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:30.046213 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-501539b5-b2ae-4583-ab8e-79508bc24a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36e10488543e47f0b786d528ba7e9c2d', 'request_id': 'req-501539b5-b2ae-4583-ab8e-79508bc24a82'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:30.046373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1268/2494] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:29 2022] PUT /identity/v3/projects/85f660610db34845b21dcebe5721eccb/users/d465a041744f45e4be3d22cffd4de349/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:30.057411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196f808-cfad-4451-8b9c-3648d401948e None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.057633 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196f808-cfad-4451-8b9c-3648d401948e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.057900 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196f808-cfad-4451-8b9c-3648d401948e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.059219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-501539b5-b2ae-4583-ab8e-79508bc24a82 admin admin] The token cache is being invalidate because project 36e10488543e47f0b786d528ba7e9c2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:30.059670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-501539b5-b2ae-4583-ab8e-79508bc24a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:30.060500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-501539b5-b2ae-4583-ab8e-79508bc24a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:30.063941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1227/2495] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:29 2022] DELETE /identity/v3/projects/36e10488543e47f0b786d528ba7e9c2d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:30.067019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8055b-1afc-4314-818b-beb5c81f040b None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.067492 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8055b-1afc-4314-818b-beb5c81f040b None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.068027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8055b-1afc-4314-818b-beb5c81f040b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.087243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-f4b8055b-1afc-4314-818b-beb5c81f040b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:30.087492 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-7196f808-cfad-4451-8b9c-3648d401948e None None] MFA Rules not processed for user `d465a041744f45e4be3d22cffd4de349`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:30.088510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f4b8055b-1afc-4314-818b-beb5c81f040b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:30.088682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-7196f808-cfad-4451-8b9c-3648d401948e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:30.149805 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1228/2496] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:30.158497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4e350-6452-477c-8551-3df508ed3352 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.159045 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4e350-6452-477c-8551-3df508ed3352 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.159396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4e350-6452-477c-8551-3df508ed3352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.165146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1269/2497] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:30.166091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196f808-cfad-4451-8b9c-3648d401948e None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.167871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-7196f808-cfad-4451-8b9c-3648d401948e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:30.179008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-bba4e350-6452-477c-8551-3df508ed3352 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:30.180236 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-bba4e350-6452-477c-8551-3df508ed3352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:30.182679 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196f808-cfad-4451-8b9c-3648d401948e None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.188426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554aa7f-f24e-42e9-b980-e4d637bcc1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.189084 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554aa7f-f24e-42e9-b980-e4d637bcc1f1 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.189578 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1229/2498] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:30.192522 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554aa7f-f24e-42e9-b980-e4d637bcc1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.192951 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554aa7f-f24e-42e9-b980-e4d637bcc1f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.193794 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554aa7f-f24e-42e9-b980-e4d637bcc1f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.193907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4e350-6452-477c-8551-3df508ed3352 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.195652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554aa7f-f24e-42e9-b980-e4d637bcc1f1 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.203505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4e350-6452-477c-8551-3df508ed3352 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.208367 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e354b-bda2-4c18-83ab-c24353667fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.209020 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e354b-bda2-4c18-83ab-c24353667fed admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.209368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e354b-bda2-4c18-83ab-c24353667fed admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.209641 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e354b-bda2-4c18-83ab-c24353667fed admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.210358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e354b-bda2-4c18-83ab-c24353667fed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:33:30.210812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e354b-bda2-4c18-83ab-c24353667fed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.212530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e354b-bda2-4c18-83ab-c24353667fed admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.225395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1230/2499] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:33:30.226868 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1270/2500] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:30 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:30.229661 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e354b-bda2-4c18-83ab-c24353667fed admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.230791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554aa7f-f24e-42e9-b980-e4d637bcc1f1 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.240756 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e354b-bda2-4c18-83ab-c24353667fed admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.296295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554aa7f-f24e-42e9-b980-e4d637bcc1f1 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.299458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49fb5a-e9b5-44bc-a552-839a86c95eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.300157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49fb5a-e9b5-44bc-a552-839a86c95eda admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.300562 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49fb5a-e9b5-44bc-a552-839a86c95eda admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.300845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49fb5a-e9b5-44bc-a552-839a86c95eda admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.300962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f78c7b-8b1a-419a-81db-e111e8850b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.301534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49fb5a-e9b5-44bc-a552-839a86c95eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.301646 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f78c7b-8b1a-419a-81db-e111e8850b63 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.301876 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f78c7b-8b1a-419a-81db-e111e8850b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.302156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f78c7b-8b1a-419a-81db-e111e8850b63 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.302880 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f78c7b-8b1a-419a-81db-e111e8850b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.304069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49fb5a-e9b5-44bc-a552-839a86c95eda admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.304829 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f78c7b-8b1a-419a-81db-e111e8850b63 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.336587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1271/2501] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:30 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:30.336791 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1231/2502] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:30 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:30.338794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49fb5a-e9b5-44bc-a552-839a86c95eda admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.338950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f78c7b-8b1a-419a-81db-e111e8850b63 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.352406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f78c7b-8b1a-419a-81db-e111e8850b63 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.406760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49fb5a-e9b5-44bc-a552-839a86c95eda admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.413112 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dae06-3582-4d93-af2b-b5ed09a63139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.413282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a585a-5fa6-450f-80e1-8c1e67dd2ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.413827 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a585a-5fa6-450f-80e1-8c1e67dd2ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.413989 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dae06-3582-4d93-af2b-b5ed09a63139 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.414116 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a585a-5fa6-450f-80e1-8c1e67dd2ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.414362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dae06-3582-4d93-af2b-b5ed09a63139 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.414647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dae06-3582-4d93-af2b-b5ed09a63139 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.414758 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a585a-5fa6-450f-80e1-8c1e67dd2ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.415393 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a585a-5fa6-450f-80e1-8c1e67dd2ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.415684 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094dae06-3582-4d93-af2b-b5ed09a63139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.417752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094dae06-3582-4d93-af2b-b5ed09a63139 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.418199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a585a-5fa6-450f-80e1-8c1e67dd2ce7 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.435938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1232/2503] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:30.437872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a585a-5fa6-450f-80e1-8c1e67dd2ce7 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.444971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1272/2504] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:30 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:30.445955 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dae06-3582-4d93-af2b-b5ed09a63139 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.456984 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a585a-5fa6-450f-80e1-8c1e67dd2ce7 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.460154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094dae06-3582-4d93-af2b-b5ed09a63139 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.462874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e221abb-3d53-4c15-a5c1-7ac25de69e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.463831 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e221abb-3d53-4c15-a5c1-7ac25de69e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.464461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e221abb-3d53-4c15-a5c1-7ac25de69e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.464799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e221abb-3d53-4c15-a5c1-7ac25de69e74 admin admin] PATH_INFO: `/v3/projects/94a348473e1a471cb7e62fa2951a4557/users/304c8d4ac48f4218a9bafcd6574a2904/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.465663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372135a-2841-44e1-bff8-f0ef6cf6733a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.466396 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372135a-2841-44e1-bff8-f0ef6cf6733a admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.466836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372135a-2841-44e1-bff8-f0ef6cf6733a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.467653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372135a-2841-44e1-bff8-f0ef6cf6733a admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.469790 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372135a-2841-44e1-bff8-f0ef6cf6733a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.471640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372135a-2841-44e1-bff8-f0ef6cf6733a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.481445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e221abb-3d53-4c15-a5c1-7ac25de69e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a348473e1a471cb7e62fa2951a4557, user_id=304c8d4ac48f4218a9bafcd6574a2904, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.483691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1273/2505] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:30.484551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e221abb-3d53-4c15-a5c1-7ac25de69e74 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.487364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372135a-2841-44e1-bff8-f0ef6cf6733a admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.497619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372135a-2841-44e1-bff8-f0ef6cf6733a admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.501166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1233/2506] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:30 2022] PUT /identity/v3/projects/94a348473e1a471cb7e62fa2951a4557/users/304c8d4ac48f4218a9bafcd6574a2904/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:30.505170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e221abb-3d53-4c15-a5c1-7ac25de69e74 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.559526 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8455436-4fcb-4e96-aa5c-382e8fe7be4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.560215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8455436-4fcb-4e96-aa5c-382e8fe7be4a admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.561161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8455436-4fcb-4e96-aa5c-382e8fe7be4a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.561652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8455436-4fcb-4e96-aa5c-382e8fe7be4a admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.562509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8455436-4fcb-4e96-aa5c-382e8fe7be4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.564147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8455436-4fcb-4e96-aa5c-382e8fe7be4a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.578587 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1274/2507] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:33:30.579338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8455436-4fcb-4e96-aa5c-382e8fe7be4a admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.581209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e221abb-3d53-4c15-a5c1-7ac25de69e74 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.586035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb439d-7efb-4128-b12f-6629d9f7a88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.586879 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb439d-7efb-4128-b12f-6629d9f7a88b admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.587209 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb439d-7efb-4128-b12f-6629d9f7a88b admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.587586 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb439d-7efb-4128-b12f-6629d9f7a88b admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.588477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb439d-7efb-4128-b12f-6629d9f7a88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.589787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8455436-4fcb-4e96-aa5c-382e8fe7be4a admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.590510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb439d-7efb-4128-b12f-6629d9f7a88b admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.594762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd88079-5b84-4509-8bfc-1106ccc27ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.595726 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd88079-5b84-4509-8bfc-1106ccc27ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.596224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd88079-5b84-4509-8bfc-1106ccc27ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.596571 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd88079-5b84-4509-8bfc-1106ccc27ea5 admin admin] PATH_INFO: `/v3/projects/3d6aeac77b6d4f1c8f912d7c5d91968f/users/ec0360cd80a040c5ae2f3f7a6ba01f75/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.601131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1234/2508] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:33:30.605334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb439d-7efb-4128-b12f-6629d9f7a88b admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.616499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd88079-5b84-4509-8bfc-1106ccc27ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6aeac77b6d4f1c8f912d7c5d91968f, user_id=ec0360cd80a040c5ae2f3f7a6ba01f75, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.621154 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb439d-7efb-4128-b12f-6629d9f7a88b admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.621416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd88079-5b84-4509-8bfc-1106ccc27ea5 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.628870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd726d5-0e90-46f9-96a4-c82acedcd6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.630090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd726d5-0e90-46f9-96a4-c82acedcd6bc admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.630455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd726d5-0e90-46f9-96a4-c82acedcd6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.630783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd726d5-0e90-46f9-96a4-c82acedcd6bc admin admin] PATH_INFO: `/v3/projects/94a348473e1a471cb7e62fa2951a4557/users/304c8d4ac48f4218a9bafcd6574a2904/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.635023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd726d5-0e90-46f9-96a4-c82acedcd6bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a348473e1a471cb7e62fa2951a4557, user_id=304c8d4ac48f4218a9bafcd6574a2904, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.638245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd726d5-0e90-46f9-96a4-c82acedcd6bc admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.642061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1275/2509] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:30 2022] PUT /identity/v3/projects/3d6aeac77b6d4f1c8f912d7c5d91968f/users/ec0360cd80a040c5ae2f3f7a6ba01f75/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:30.647637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdbfeb-072b-4237-a855-d69c8f537565 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.647977 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdbfeb-072b-4237-a855-d69c8f537565 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.648282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdbfeb-072b-4237-a855-d69c8f537565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.658645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1235/2510] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:30 2022] PUT /identity/v3/projects/94a348473e1a471cb7e62fa2951a4557/users/304c8d4ac48f4218a9bafcd6574a2904/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:30.668890 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e8589-e6b7-4167-b45a-6a13a21e241a None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.669194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e8589-e6b7-4167-b45a-6a13a21e241a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.669472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e8589-e6b7-4167-b45a-6a13a21e241a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.672638 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-accdbfeb-072b-4237-a855-d69c8f537565 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:30.674103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-accdbfeb-072b-4237-a855-d69c8f537565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:30.710932 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d45e8589-e6b7-4167-b45a-6a13a21e241a None None] MFA Rules not processed for user `304c8d4ac48f4218a9bafcd6574a2904`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:30.712392 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d45e8589-e6b7-4167-b45a-6a13a21e241a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:30.779698 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1276/2511] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:30.793100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdbfeb-072b-4237-a855-d69c8f537565 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.794595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-accdbfeb-072b-4237-a855-d69c8f537565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:30.830630 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1236/2512] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:30.844874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e8589-e6b7-4167-b45a-6a13a21e241a None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:30.847533 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d45e8589-e6b7-4167-b45a-6a13a21e241a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:30.911443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdbfeb-072b-4237-a855-d69c8f537565 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.916489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6f365-db9e-4ebb-8b1b-bb4a5c94cfca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.917360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6f365-db9e-4ebb-8b1b-bb4a5c94cfca service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.917618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6f365-db9e-4ebb-8b1b-bb4a5c94cfca service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.917993 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6f365-db9e-4ebb-8b1b-bb4a5c94cfca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.929504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6f365-db9e-4ebb-8b1b-bb4a5c94cfca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:30.931354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6f365-db9e-4ebb-8b1b-bb4a5c94cfca service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:30.938166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e8589-e6b7-4167-b45a-6a13a21e241a None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:30.943009 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881da19-69c4-4628-b4fb-4884ce3c42c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:30.943252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1277/2513] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:33:30.944150 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881da19-69c4-4628-b4fb-4884ce3c42c3 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:30.944794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881da19-69c4-4628-b4fb-4884ce3c42c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:30.945239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881da19-69c4-4628-b4fb-4884ce3c42c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:30.947487 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b881da19-69c4-4628-b4fb-4884ce3c42c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:31.022021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881da19-69c4-4628-b4fb-4884ce3c42c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:31.024062 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881da19-69c4-4628-b4fb-4884ce3c42c3 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:31.035390 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1237/2514] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:33:31.971932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0131d1e-4b29-47a4-a612-42afe5dd78d8 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:31.972979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0131d1e-4b29-47a4-a612-42afe5dd78d8 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:31.973835 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0131d1e-4b29-47a4-a612-42afe5dd78d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:32.004184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-d0131d1e-4b29-47a4-a612-42afe5dd78d8 None None] MFA Rules not processed for user `ec0360cd80a040c5ae2f3f7a6ba01f75`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:32.005517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d0131d1e-4b29-47a4-a612-42afe5dd78d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:32.082111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1278/2515] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:32.095227 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881da19-69c4-4628-b4fb-4884ce3c42c3 service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:32.101298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b881da19-69c4-4628-b4fb-4884ce3c42c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:32.115695 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881da19-69c4-4628-b4fb-4884ce3c42c3 service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:32.120008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850c83b-ddc9-44f6-bb6b-24ac86866372 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:32.120817 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850c83b-ddc9-44f6-bb6b-24ac86866372 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:32.121212 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850c83b-ddc9-44f6-bb6b-24ac86866372 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:32.121365 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850c83b-ddc9-44f6-bb6b-24ac86866372 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:32.132089 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850c83b-ddc9-44f6-bb6b-24ac86866372 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:32.133531 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850c83b-ddc9-44f6-bb6b-24ac86866372 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:32.146527 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1238/2516] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:32.740860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0131d1e-4b29-47a4-a612-42afe5dd78d8 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:32.756283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0131d1e-4b29-47a4-a612-42afe5dd78d8 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:32.761421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf51c28-f6ce-4b30-a7a3-e1edf52798d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:32.762238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf51c28-f6ce-4b30-a7a3-e1edf52798d0 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:32.762625 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf51c28-f6ce-4b30-a7a3-e1edf52798d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:32.763036 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf51c28-f6ce-4b30-a7a3-e1edf52798d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:32.867790 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf51c28-f6ce-4b30-a7a3-e1edf52798d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:32.869286 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf51c28-f6ce-4b30-a7a3-e1edf52798d0 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:32.881111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1279/2517] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:33.565810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc7a8c-c6a1-472d-8f0e-7500064b51e8 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:33.566269 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc7a8c-c6a1-472d-8f0e-7500064b51e8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:33.566368 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc7a8c-c6a1-472d-8f0e-7500064b51e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:33.596751 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-1dbc7a8c-c6a1-472d-8f0e-7500064b51e8 None None] MFA Rules not processed for user `ec0360cd80a040c5ae2f3f7a6ba01f75`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:33.597998 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1dbc7a8c-c6a1-472d-8f0e-7500064b51e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:33.616228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1239/2518] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:33.743870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf51c28-f6ce-4b30-a7a3-e1edf52798d0 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:33.759448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf51c28-f6ce-4b30-a7a3-e1edf52798d0 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:33.766398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04461ed-cfdc-4195-8968-80040c26de87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:33.767060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04461ed-cfdc-4195-8968-80040c26de87 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:33.767340 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04461ed-cfdc-4195-8968-80040c26de87 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:33.767749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04461ed-cfdc-4195-8968-80040c26de87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:33.794508 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04461ed-cfdc-4195-8968-80040c26de87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:33.796622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04461ed-cfdc-4195-8968-80040c26de87 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:33.824261 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1280/2519] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:37.797297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc7a8c-c6a1-472d-8f0e-7500064b51e8 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:37.815060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc7a8c-c6a1-472d-8f0e-7500064b51e8 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:37.820270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:37.821142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:37.821448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:37.821743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] PATH_INFO: `/v3/users/ee43672d56b248889eff989eedda263d` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:37.823468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee43672d56b248889eff989eedda263d)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:37.825430 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:37.881359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee43672d56b248889eff989eedda263d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:37.882013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee43672d56b248889eff989eedda263d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:37.882615 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee43672d56b248889eff989eedda263d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:37.891666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee43672d56b248889eff989eedda263d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:37.901255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee43672d56b248889eff989eedda263d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:37.932440 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1240/2520] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:37 2022] DELETE /identity/v3/users/ee43672d56b248889eff989eedda263d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:37.936420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bfde1-8038-47ce-a292-784da0ae00d9 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:37.937776 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bfde1-8038-47ce-a292-784da0ae00d9 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:37.938222 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bfde1-8038-47ce-a292-784da0ae00d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:37.957521 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-892bfde1-8038-47ce-a292-784da0ae00d9 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:37.958761 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-892bfde1-8038-47ce-a292-784da0ae00d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:38.048060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1281/2521] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:38.059420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:38.061819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:38.157818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da37c59-5db9-4a2e-93c3-19bcb70eecac admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:38.162075 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e16e3e-7573-4264-beec-a16b73c268c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:38.162923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e16e3e-7573-4264-beec-a16b73c268c4 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:38.163207 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e16e3e-7573-4264-beec-a16b73c268c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:38.163492 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e16e3e-7573-4264-beec-a16b73c268c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:38.173921 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e16e3e-7573-4264-beec-a16b73c268c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:38.175397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e16e3e-7573-4264-beec-a16b73c268c4 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:38.186226 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1241/2522] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:38.470393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bfde1-8038-47ce-a292-784da0ae00d9 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:38.472634 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-892bfde1-8038-47ce-a292-784da0ae00d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:38.486402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bfde1-8038-47ce-a292-784da0ae00d9 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:38.492562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:38.493717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:38.494109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:38.494889 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] PATH_INFO: `/v3/projects/90a2839af354461eb78797a1ca8291c6` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:38.497364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a2839af354461eb78797a1ca8291c6)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:38.500715 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:38.548504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a2839af354461eb78797a1ca8291c6', 'request_id': 'req-e0003194-54f6-4f2d-8d2f-19361422c5a8'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:38.553845 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a2839af354461eb78797a1ca8291c6', 'request_id': 'req-e0003194-54f6-4f2d-8d2f-19361422c5a8'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:38.562307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a2839af354461eb78797a1ca8291c6', 'request_id': 'req-e0003194-54f6-4f2d-8d2f-19361422c5a8'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:38.573314 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] The token cache is being invalidate because project 90a2839af354461eb78797a1ca8291c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:38.573605 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:38.574254 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:38.576617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1282/2523] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:38 2022] DELETE /identity/v3/projects/90a2839af354461eb78797a1ca8291c6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:38.580552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e16e3e-7573-4264-beec-a16b73c268c4 service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:38.590178 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e16e3e-7573-4264-beec-a16b73c268c4 service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:38.646604 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:38.647657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:38.648050 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:38.648381 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] PATH_INFO: `/v3/users/d465a041744f45e4be3d22cffd4de349` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:38.649640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d465a041744f45e4be3d22cffd4de349)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:38.651629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:38.698778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd465a041744f45e4be3d22cffd4de349'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:38.699523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd465a041744f45e4be3d22cffd4de349'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:38.700205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd465a041744f45e4be3d22cffd4de349'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:38.709685 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd465a041744f45e4be3d22cffd4de349'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:38.717041 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd465a041744f45e4be3d22cffd4de349'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:38.738764 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1242/2524] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:38 2022] DELETE /identity/v3/users/d465a041744f45e4be3d22cffd4de349 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:39.028426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:39.031831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:39.053280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0003194-54f6-4f2d-8d2f-19361422c5a8 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:39.114538 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:39.115554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:39.115961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:39.116344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] PATH_INFO: `/v3/projects/85f660610db34845b21dcebe5721eccb` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:39.117549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] RBAC: Authorizing `identity:delete_project(project_id=85f660610db34845b21dcebe5721eccb)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:39.119779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:39.177084 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85f660610db34845b21dcebe5721eccb', 'request_id': 'req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.183931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85f660610db34845b21dcebe5721eccb', 'request_id': 'req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.196701 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85f660610db34845b21dcebe5721eccb', 'request_id': 'req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.208859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] The token cache is being invalidate because project 85f660610db34845b21dcebe5721eccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:39.209144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.209958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.212201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1283/2525] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:39 2022] DELETE /identity/v3/projects/85f660610db34845b21dcebe5721eccb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:39.216684 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:39.219238 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:39.233601 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285d8ac-706e-45bc-8f5b-13632a5b408d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:39.302069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bebde9-e2c9-4c6a-bff0-bdc697f46128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:39.303090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bebde9-e2c9-4c6a-bff0-bdc697f46128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:39.303474 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bebde9-e2c9-4c6a-bff0-bdc697f46128 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:39.303864 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bebde9-e2c9-4c6a-bff0-bdc697f46128 admin admin] PATH_INFO: `/v3/users/304c8d4ac48f4218a9bafcd6574a2904` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:39.305185 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bebde9-e2c9-4c6a-bff0-bdc697f46128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=304c8d4ac48f4218a9bafcd6574a2904)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:39.307454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bebde9-e2c9-4c6a-bff0-bdc697f46128 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:39.356144 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-13bebde9-e2c9-4c6a-bff0-bdc697f46128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '304c8d4ac48f4218a9bafcd6574a2904'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.360611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-13bebde9-e2c9-4c6a-bff0-bdc697f46128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '304c8d4ac48f4218a9bafcd6574a2904'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.361585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-13bebde9-e2c9-4c6a-bff0-bdc697f46128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '304c8d4ac48f4218a9bafcd6574a2904'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.370644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-13bebde9-e2c9-4c6a-bff0-bdc697f46128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '304c8d4ac48f4218a9bafcd6574a2904'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.384208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-13bebde9-e2c9-4c6a-bff0-bdc697f46128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '304c8d4ac48f4218a9bafcd6574a2904'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.407804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1243/2526] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:39 2022] DELETE /identity/v3/users/304c8d4ac48f4218a9bafcd6574a2904 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:39.702647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:39.704386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:39.718489 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7e29a-3895-4a1f-9cc3-f9bf271bd68e admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:39.786599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2007da-d34f-41c1-8706-2f14c6feab81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:39.787465 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2007da-d34f-41c1-8706-2f14c6feab81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:39.787745 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2007da-d34f-41c1-8706-2f14c6feab81 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:39.788045 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2007da-d34f-41c1-8706-2f14c6feab81 admin admin] PATH_INFO: `/v3/projects/94a348473e1a471cb7e62fa2951a4557` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:39.789249 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2007da-d34f-41c1-8706-2f14c6feab81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94a348473e1a471cb7e62fa2951a4557)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:39.791252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2007da-d34f-41c1-8706-2f14c6feab81 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:39.841497 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-cc2007da-d34f-41c1-8706-2f14c6feab81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94a348473e1a471cb7e62fa2951a4557', 'request_id': 'req-cc2007da-d34f-41c1-8706-2f14c6feab81'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.847293 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-cc2007da-d34f-41c1-8706-2f14c6feab81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94a348473e1a471cb7e62fa2951a4557', 'request_id': 'req-cc2007da-d34f-41c1-8706-2f14c6feab81'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.855040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-cc2007da-d34f-41c1-8706-2f14c6feab81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94a348473e1a471cb7e62fa2951a4557', 'request_id': 'req-cc2007da-d34f-41c1-8706-2f14c6feab81'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.866994 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-cc2007da-d34f-41c1-8706-2f14c6feab81 admin admin] The token cache is being invalidate because project 94a348473e1a471cb7e62fa2951a4557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:39.867282 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-cc2007da-d34f-41c1-8706-2f14c6feab81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.867985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-cc2007da-d34f-41c1-8706-2f14c6feab81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:39.869536 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1284/2527] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:39 2022] DELETE /identity/v3/projects/94a348473e1a471cb7e62fa2951a4557 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:39.874802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168728b-8b2d-405e-9ffc-b0574021c098 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:39.875102 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168728b-8b2d-405e-9ffc-b0574021c098 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:39.875385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168728b-8b2d-405e-9ffc-b0574021c098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:39.894922 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-e168728b-8b2d-405e-9ffc-b0574021c098 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:39.896229 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e168728b-8b2d-405e-9ffc-b0574021c098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:40.004523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1244/2528] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:40.007817 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f0b81-e32b-48a5-a127-378819fdbcdb None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:40.008206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f0b81-e32b-48a5-a127-378819fdbcdb None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:40.008554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f0b81-e32b-48a5-a127-378819fdbcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:40.032551 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-ec9f0b81-e32b-48a5-a127-378819fdbcdb None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:40.035266 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ec9f0b81-e32b-48a5-a127-378819fdbcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:40.061341 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168728b-8b2d-405e-9ffc-b0574021c098 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:40.070217 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1285/2529] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:40.076624 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168728b-8b2d-405e-9ffc-b0574021c098 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:40.135465 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982144d4-8a9c-4824-8e1c-2326431faac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:40.140197 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982144d4-8a9c-4824-8e1c-2326431faac9 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:40.141414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982144d4-8a9c-4824-8e1c-2326431faac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:40.141804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982144d4-8a9c-4824-8e1c-2326431faac9 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:40.143754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982144d4-8a9c-4824-8e1c-2326431faac9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:33:40.143754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982144d4-8a9c-4824-8e1c-2326431faac9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:40.158049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982144d4-8a9c-4824-8e1c-2326431faac9 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:40.181941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f0b81-e32b-48a5-a127-378819fdbcdb None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:40.195580 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1245/2530] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:33:40.200000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f0b81-e32b-48a5-a127-378819fdbcdb None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:40.208738 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0929994-e1ff-4152-a0d7-c05af615ed83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:40.209706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0929994-e1ff-4152-a0d7-c05af615ed83 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:40.210298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0929994-e1ff-4152-a0d7-c05af615ed83 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:40.211158 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0929994-e1ff-4152-a0d7-c05af615ed83 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:40.212391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0929994-e1ff-4152-a0d7-c05af615ed83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:40.215690 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0929994-e1ff-4152-a0d7-c05af615ed83 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:40.281726 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1286/2531] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:40 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:40.286019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982144d4-8a9c-4824-8e1c-2326431faac9 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:40.297650 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982144d4-8a9c-4824-8e1c-2326431faac9 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:40.427420 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0df1d5-5937-4585-b8ce-1b67e55fcf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:40.429191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0df1d5-5937-4585-b8ce-1b67e55fcf4d admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:40.430370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0df1d5-5937-4585-b8ce-1b67e55fcf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:40.430823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0df1d5-5937-4585-b8ce-1b67e55fcf4d admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:40.431810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0df1d5-5937-4585-b8ce-1b67e55fcf4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:40.442644 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0df1d5-5937-4585-b8ce-1b67e55fcf4d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:40.508899 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0929994-e1ff-4152-a0d7-c05af615ed83 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:40.521160 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1246/2532] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:40 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:40.526551 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0929994-e1ff-4152-a0d7-c05af615ed83 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:40.536887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470b83a-9c40-4cba-b630-293a58c1cad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:40.538592 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470b83a-9c40-4cba-b630-293a58c1cad8 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:40.539013 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470b83a-9c40-4cba-b630-293a58c1cad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:40.539383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470b83a-9c40-4cba-b630-293a58c1cad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:40.540570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470b83a-9c40-4cba-b630-293a58c1cad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:40.542877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470b83a-9c40-4cba-b630-293a58c1cad8 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:40.555736 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1287/2533] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:40.559834 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0df1d5-5937-4585-b8ce-1b67e55fcf4d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:40.574123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0df1d5-5937-4585-b8ce-1b67e55fcf4d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:40.580969 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef4d30-ab56-425f-b949-b505ea1de131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:40.581671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef4d30-ab56-425f-b949-b505ea1de131 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:40.581923 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef4d30-ab56-425f-b949-b505ea1de131 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:40.582201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef4d30-ab56-425f-b949-b505ea1de131 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:40.583106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef4d30-ab56-425f-b949-b505ea1de131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:40.584752 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef4d30-ab56-425f-b949-b505ea1de131 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:40.599859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1247/2534] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:40.604170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470b83a-9c40-4cba-b630-293a58c1cad8 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:40.616013 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470b83a-9c40-4cba-b630-293a58c1cad8 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:40.624240 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd01d3-f1fb-4e89-b812-7fa2e88fa559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:40.625273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd01d3-f1fb-4e89-b812-7fa2e88fa559 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:40.625599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd01d3-f1fb-4e89-b812-7fa2e88fa559 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:40.626246 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd01d3-f1fb-4e89-b812-7fa2e88fa559 admin admin] PATH_INFO: `/v3/projects/e30fec6f567c4058bf9d445e96cecb29/users/30970d0f6a70463ca566586955310ea6/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:40.644368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd01d3-f1fb-4e89-b812-7fa2e88fa559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e30fec6f567c4058bf9d445e96cecb29, user_id=30970d0f6a70463ca566586955310ea6, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:40.648160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd01d3-f1fb-4e89-b812-7fa2e88fa559 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:40.665648 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1288/2535] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:40 2022] PUT /identity/v3/projects/e30fec6f567c4058bf9d445e96cecb29/users/30970d0f6a70463ca566586955310ea6/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:40.678097 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644b904-51a8-47bb-9d04-2b41ccd25b20 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:40.678542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644b904-51a8-47bb-9d04-2b41ccd25b20 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:40.678863 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644b904-51a8-47bb-9d04-2b41ccd25b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:40.709154 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-4644b904-51a8-47bb-9d04-2b41ccd25b20 None None] MFA Rules not processed for user `30970d0f6a70463ca566586955310ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:40.710713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-4644b904-51a8-47bb-9d04-2b41ccd25b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:40.798823 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1248/2536] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:40.803325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd01d3-f1fb-4e89-b812-7fa2e88fa559 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:40.894183 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd01d3-f1fb-4e89-b812-7fa2e88fa559 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:40.899751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d139c4-4dea-400d-ab90-21bca2fee673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:40.900601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d139c4-4dea-400d-ab90-21bca2fee673 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:40.900887 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d139c4-4dea-400d-ab90-21bca2fee673 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:40.901166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d139c4-4dea-400d-ab90-21bca2fee673 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:40.902028 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d139c4-4dea-400d-ab90-21bca2fee673 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:40.904198 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d139c4-4dea-400d-ab90-21bca2fee673 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:40.970406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1289/2537] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:40 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:40.974146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644b904-51a8-47bb-9d04-2b41ccd25b20 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:40.989755 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644b904-51a8-47bb-9d04-2b41ccd25b20 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:41.056310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048d4ff-31dd-4230-ade1-dad20c89e3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:41.057007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048d4ff-31dd-4230-ade1-dad20c89e3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:41.057298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048d4ff-31dd-4230-ade1-dad20c89e3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:41.057588 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048d4ff-31dd-4230-ade1-dad20c89e3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:41.058267 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048d4ff-31dd-4230-ade1-dad20c89e3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:41.060217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048d4ff-31dd-4230-ade1-dad20c89e3a8 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:41.087435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1249/2538] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:40 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:41.092411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d139c4-4dea-400d-ab90-21bca2fee673 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:41.104499 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d139c4-4dea-400d-ab90-21bca2fee673 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:41.110917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1daf0-9574-4766-906b-e24ebff765de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:41.111707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1daf0-9574-4766-906b-e24ebff765de admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:41.112140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1daf0-9574-4766-906b-e24ebff765de admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:41.112526 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1daf0-9574-4766-906b-e24ebff765de admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:41.113693 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1daf0-9574-4766-906b-e24ebff765de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:41.117368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1daf0-9574-4766-906b-e24ebff765de admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:41.140327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1290/2539] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:41.142944 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048d4ff-31dd-4230-ade1-dad20c89e3a8 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:41.166602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048d4ff-31dd-4230-ade1-dad20c89e3a8 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:41.184354 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994cd66-0c81-4fc9-af18-38d1fa2bc488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:41.186200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994cd66-0c81-4fc9-af18-38d1fa2bc488 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:41.186681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994cd66-0c81-4fc9-af18-38d1fa2bc488 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:41.187166 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994cd66-0c81-4fc9-af18-38d1fa2bc488 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:41.188136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1994cd66-0c81-4fc9-af18-38d1fa2bc488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:41.190637 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1994cd66-0c81-4fc9-af18-38d1fa2bc488 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:41.205295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1250/2540] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:41.208944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1daf0-9574-4766-906b-e24ebff765de admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:41.225401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1daf0-9574-4766-906b-e24ebff765de admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:41.233035 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41da22-bacb-49ee-a87b-e7edab504112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:41.234433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41da22-bacb-49ee-a87b-e7edab504112 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:41.234927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41da22-bacb-49ee-a87b-e7edab504112 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:41.235266 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41da22-bacb-49ee-a87b-e7edab504112 admin admin] PATH_INFO: `/v3/projects/c29d8ea15635473b9977f455ac5437bb/users/1a60bd63cac44fd4b5dc3c9372515bbe/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:41.263834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41da22-bacb-49ee-a87b-e7edab504112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29d8ea15635473b9977f455ac5437bb, user_id=1a60bd63cac44fd4b5dc3c9372515bbe, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:41.267194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41da22-bacb-49ee-a87b-e7edab504112 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:41.282487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1291/2541] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:41 2022] PUT /identity/v3/projects/c29d8ea15635473b9977f455ac5437bb/users/1a60bd63cac44fd4b5dc3c9372515bbe/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:41.292941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa72e1e-50b5-421e-85a9-3a6cd6301dc0 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:41.293430 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa72e1e-50b5-421e-85a9-3a6cd6301dc0 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:41.293754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa72e1e-50b5-421e-85a9-3a6cd6301dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:41.323853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-3fa72e1e-50b5-421e-85a9-3a6cd6301dc0 None None] MFA Rules not processed for user `1a60bd63cac44fd4b5dc3c9372515bbe`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:41.325221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3fa72e1e-50b5-421e-85a9-3a6cd6301dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:41.401358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1251/2542] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:41.405859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41da22-bacb-49ee-a87b-e7edab504112 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:41.433888 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa72e1e-50b5-421e-85a9-3a6cd6301dc0 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:41.436312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3fa72e1e-50b5-421e-85a9-3a6cd6301dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:41.536632 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa72e1e-50b5-421e-85a9-3a6cd6301dc0 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:41.542755 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:41.544090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:41.544560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:41.545298 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] PATH_INFO: `/v3/users/dc01367aaf7742d8abdd0fdc4de64f6e` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:41.546781 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc01367aaf7742d8abdd0fdc4de64f6e)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:41.549499 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:41.578042 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41da22-bacb-49ee-a87b-e7edab504112 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:41.583555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977de06d-10c0-42f1-abee-a706f28f4110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:41.584289 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977de06d-10c0-42f1-abee-a706f28f4110 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:41.584573 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977de06d-10c0-42f1-abee-a706f28f4110 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:41.584852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977de06d-10c0-42f1-abee-a706f28f4110 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:41.586176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977de06d-10c0-42f1-abee-a706f28f4110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:41.588508 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977de06d-10c0-42f1-abee-a706f28f4110 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:41.637546 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1292/2543] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:41 2022] POST /identity/v3/projects => generated 385 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:41.647017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc01367aaf7742d8abdd0fdc4de64f6e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:41.647199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977de06d-10c0-42f1-abee-a706f28f4110 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:41.648247 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc01367aaf7742d8abdd0fdc4de64f6e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:41.649091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc01367aaf7742d8abdd0fdc4de64f6e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:41.664031 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc01367aaf7742d8abdd0fdc4de64f6e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:41.671930 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc01367aaf7742d8abdd0fdc4de64f6e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:41.692168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1252/2544] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:41 2022] DELETE /identity/v3/users/dc01367aaf7742d8abdd0fdc4de64f6e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:41.718179 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977de06d-10c0-42f1-abee-a706f28f4110 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:41.812818 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13010b2d-471a-45a3-81fe-815566d0d934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:41.813717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13010b2d-471a-45a3-81fe-815566d0d934 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:41.813857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13010b2d-471a-45a3-81fe-815566d0d934 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:41.814063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13010b2d-471a-45a3-81fe-815566d0d934 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:41.814760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13010b2d-471a-45a3-81fe-815566d0d934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:41.816748 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13010b2d-471a-45a3-81fe-815566d0d934 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:41.849359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1293/2545] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:41 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:41.850556 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:41.852000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:41.877180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657c578-c55a-46c6-ae90-08d53a3451fa admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:41.882629 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2701f-ff1f-484f-b964-2c3adef5b157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:41.883286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2701f-ff1f-484f-b964-2c3adef5b157 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:41.885068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2701f-ff1f-484f-b964-2c3adef5b157 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:41.885068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2701f-ff1f-484f-b964-2c3adef5b157 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:41.885392 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2701f-ff1f-484f-b964-2c3adef5b157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:41.887310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2701f-ff1f-484f-b964-2c3adef5b157 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:41.901413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1253/2546] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:41.905331 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13010b2d-471a-45a3-81fe-815566d0d934 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:41.920218 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13010b2d-471a-45a3-81fe-815566d0d934 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:41.947249 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e30e1d-52fa-4c31-8f3a-7086bd1acb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:41.947962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e30e1d-52fa-4c31-8f3a-7086bd1acb7a admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:41.948341 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e30e1d-52fa-4c31-8f3a-7086bd1acb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:41.948619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e30e1d-52fa-4c31-8f3a-7086bd1acb7a admin admin] PATH_INFO: `/v3/projects/c6c66198e48543b7ae9e0dfa0ca320a9/users/ecedc4e5ddb643e0b0852dc72cfdcc8c/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:41.950267 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2701f-ff1f-484f-b964-2c3adef5b157 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:41.955075 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5ae2701f-ff1f-484f-b964-2c3adef5b157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:41.973876 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e30e1d-52fa-4c31-8f3a-7086bd1acb7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c66198e48543b7ae9e0dfa0ca320a9, user_id=ecedc4e5ddb643e0b0852dc72cfdcc8c, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:41.978359 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e30e1d-52fa-4c31-8f3a-7086bd1acb7a admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:41.989177 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2701f-ff1f-484f-b964-2c3adef5b157 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:41.995462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:41.996634 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:41.997095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:41.997411 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] PATH_INFO: `/v3/projects/c8cbe32e8ede404fa3d19537256606e1` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:42.000069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8cbe32e8ede404fa3d19537256606e1)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:42.007799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:42.024105 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1294/2547] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:41 2022] PUT /identity/v3/projects/c6c66198e48543b7ae9e0dfa0ca320a9/users/ecedc4e5ddb643e0b0852dc72cfdcc8c/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:42.025287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e30e1d-52fa-4c31-8f3a-7086bd1acb7a admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:42.073925 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8cbe32e8ede404fa3d19537256606e1', 'request_id': 'req-b2a41ee5-6565-4639-bde1-fe56c900e1e9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.084722 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8cbe32e8ede404fa3d19537256606e1', 'request_id': 'req-b2a41ee5-6565-4639-bde1-fe56c900e1e9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.091228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8cbe32e8ede404fa3d19537256606e1', 'request_id': 'req-b2a41ee5-6565-4639-bde1-fe56c900e1e9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.104088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e30e1d-52fa-4c31-8f3a-7086bd1acb7a admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:42.111157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] The token cache is being invalidate because project c8cbe32e8ede404fa3d19537256606e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:42.111592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.112872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.117278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1254/2548] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:41 2022] DELETE /identity/v3/projects/c8cbe32e8ede404fa3d19537256606e1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:42.128983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:42.130450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:42.166485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f8a50-b51d-4962-8275-3c22373cbc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:42.167214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f8a50-b51d-4962-8275-3c22373cbc17 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:42.167597 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f8a50-b51d-4962-8275-3c22373cbc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:42.168009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f8a50-b51d-4962-8275-3c22373cbc17 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:42.168930 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f8a50-b51d-4962-8275-3c22373cbc17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:42.171233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f8a50-b51d-4962-8275-3c22373cbc17 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:42.188735 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1295/2549] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:42.189737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f8a50-b51d-4962-8275-3c22373cbc17 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:42.192068 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8d0f8a50-b51d-4962-8275-3c22373cbc17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:42.207958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f8a50-b51d-4962-8275-3c22373cbc17 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:42.213476 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb2949-5982-43ac-b571-795979732af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:42.214763 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb2949-5982-43ac-b571-795979732af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:42.215525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb2949-5982-43ac-b571-795979732af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:42.216249 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb2949-5982-43ac-b571-795979732af1 admin admin] PATH_INFO: `/v3/projects/c6c66198e48543b7ae9e0dfa0ca320a9/users/ecedc4e5ddb643e0b0852dc72cfdcc8c/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:42.219312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb2949-5982-43ac-b571-795979732af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c66198e48543b7ae9e0dfa0ca320a9, user_id=ecedc4e5ddb643e0b0852dc72cfdcc8c, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:42.222371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb2949-5982-43ac-b571-795979732af1 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:42.233862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1296/2550] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:42 2022] PUT /identity/v3/projects/c6c66198e48543b7ae9e0dfa0ca320a9/users/ecedc4e5ddb643e0b0852dc72cfdcc8c/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:42.239423 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a41ee5-6565-4639-bde1-fe56c900e1e9 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:42.244930 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f3cfc-7c25-4ece-9899-017f5f7dd510 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:42.245460 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f3cfc-7c25-4ece-9899-017f5f7dd510 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:42.245845 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f3cfc-7c25-4ece-9899-017f5f7dd510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:42.277065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c42f3cfc-7c25-4ece-9899-017f5f7dd510 None None] MFA Rules not processed for user `ecedc4e5ddb643e0b0852dc72cfdcc8c`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:42.278911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c42f3cfc-7c25-4ece-9899-017f5f7dd510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:42.318192 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:42.319352 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:42.319694 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:42.320064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] PATH_INFO: `/v3/users/bb4d0dd751bb4703a48a4e5066085e60` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:42.321365 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb4d0dd751bb4703a48a4e5066085e60)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:42.323362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:42.362257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1297/2551] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:42.375238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f3cfc-7c25-4ece-9899-017f5f7dd510 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:42.376908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c42f3cfc-7c25-4ece-9899-017f5f7dd510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:42.377233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb4d0dd751bb4703a48a4e5066085e60'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.378186 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb4d0dd751bb4703a48a4e5066085e60'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.378993 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb4d0dd751bb4703a48a4e5066085e60'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.390908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb4d0dd751bb4703a48a4e5066085e60'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.398267 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb4d0dd751bb4703a48a4e5066085e60'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.421676 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1255/2552] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:42 2022] DELETE /identity/v3/users/bb4d0dd751bb4703a48a4e5066085e60 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:42.471060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f3cfc-7c25-4ece-9899-017f5f7dd510 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:42.520416 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb93b9b-21d3-4270-95e3-73cef8b2e8a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:42.521182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb93b9b-21d3-4270-95e3-73cef8b2e8a7 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:42.521397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb93b9b-21d3-4270-95e3-73cef8b2e8a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:42.521990 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb93b9b-21d3-4270-95e3-73cef8b2e8a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:42.524029 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4bb93b9b-21d3-4270-95e3-73cef8b2e8a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:42.586941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb93b9b-21d3-4270-95e3-73cef8b2e8a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:42.588420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb93b9b-21d3-4270-95e3-73cef8b2e8a7 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:42.598579 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1298/2553] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:33:42.602501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:42.603838 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:42.693129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418be4e-86ba-44c6-9ca2-11dc7d13d1f2 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:42.697746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:42.698884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:42.700175 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:42.700625 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] PATH_INFO: `/v3/projects/04dd4b39fd594b19a4fa84188cf31fd4` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:42.703379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] RBAC: Authorizing `identity:delete_project(project_id=04dd4b39fd594b19a4fa84188cf31fd4)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:42.706072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:42.787920 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04dd4b39fd594b19a4fa84188cf31fd4', 'request_id': 'req-55bb0268-583f-4014-802d-11659554c78d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.800453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04dd4b39fd594b19a4fa84188cf31fd4', 'request_id': 'req-55bb0268-583f-4014-802d-11659554c78d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.800965 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb93b9b-21d3-4270-95e3-73cef8b2e8a7 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:42.802980 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4bb93b9b-21d3-4270-95e3-73cef8b2e8a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:42.809189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04dd4b39fd594b19a4fa84188cf31fd4', 'request_id': 'req-55bb0268-583f-4014-802d-11659554c78d'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.830716 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] The token cache is being invalidate because project 04dd4b39fd594b19a4fa84188cf31fd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:42.831206 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.832280 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:42.836153 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1256/2554] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:42 2022] DELETE /identity/v3/projects/04dd4b39fd594b19a4fa84188cf31fd4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:42.907169 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb93b9b-21d3-4270-95e3-73cef8b2e8a7 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:43.037657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b4de9-5d06-4c3e-a9b3-20f14f181753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:43.038548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b4de9-5d06-4c3e-a9b3-20f14f181753 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:43.038940 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b4de9-5d06-4c3e-a9b3-20f14f181753 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:43.039225 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b4de9-5d06-4c3e-a9b3-20f14f181753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:43.049768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-698b4de9-5d06-4c3e-a9b3-20f14f181753 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:43.124663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b4de9-5d06-4c3e-a9b3-20f14f181753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:43.126022 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b4de9-5d06-4c3e-a9b3-20f14f181753 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:43.136527 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1299/2555] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:44.182493 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:44.183906 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:44.244551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb0268-583f-4014-802d-11659554c78d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:44.250017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7f895-d779-4731-ad70-5a9a80d82be1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:44.250886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7f895-d779-4731-ad70-5a9a80d82be1 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:44.251165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7f895-d779-4731-ad70-5a9a80d82be1 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:44.251442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7f895-d779-4731-ad70-5a9a80d82be1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:44.253423 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e3b7f895-d779-4731-ad70-5a9a80d82be1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:44.321135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7f895-d779-4731-ad70-5a9a80d82be1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:44.322729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7f895-d779-4731-ad70-5a9a80d82be1 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:44.334069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1257/2556] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:44.929361 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b4de9-5d06-4c3e-a9b3-20f14f181753 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:44.931414 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-698b4de9-5d06-4c3e-a9b3-20f14f181753 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:44.943567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b4de9-5d06-4c3e-a9b3-20f14f181753 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:44.947636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4d6fd-ca55-49e8-ae45-bb03fab16647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:44.948983 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4d6fd-ca55-49e8-ae45-bb03fab16647 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:44.949368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4d6fd-ca55-49e8-ae45-bb03fab16647 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:44.949773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4d6fd-ca55-49e8-ae45-bb03fab16647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:44.952783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-12d4d6fd-ca55-49e8-ae45-bb03fab16647 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:45.023810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4d6fd-ca55-49e8-ae45-bb03fab16647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:45.025352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4d6fd-ca55-49e8-ae45-bb03fab16647 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:45.037164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1300/2557] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:46.799612 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7f895-d779-4731-ad70-5a9a80d82be1 service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:46.801169 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e3b7f895-d779-4731-ad70-5a9a80d82be1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:46.815447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7f895-d779-4731-ad70-5a9a80d82be1 service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:46.820189 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:46.821063 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:46.821456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:46.821643 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] PATH_INFO: `/v3/users/4f8f6c77f898475e8acf1c4042c3928b` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:46.822857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f8f6c77f898475e8acf1c4042c3928b)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:46.824841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:46.874847 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f8f6c77f898475e8acf1c4042c3928b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:46.875543 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f8f6c77f898475e8acf1c4042c3928b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:46.876184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f8f6c77f898475e8acf1c4042c3928b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:46.884473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f8f6c77f898475e8acf1c4042c3928b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:46.891700 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f8f6c77f898475e8acf1c4042c3928b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:46.910042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1258/2558] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:46 2022] DELETE /identity/v3/users/4f8f6c77f898475e8acf1c4042c3928b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:47.162322 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4d6fd-ca55-49e8-ae45-bb03fab16647 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:47.173626 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4d6fd-ca55-49e8-ae45-bb03fab16647 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:47.233161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:47.234028 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:47.234343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:47.234619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] PATH_INFO: `/v3/projects/5e56ba7f9cc64897a1d95ac15061e9e4` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:47.235816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e56ba7f9cc64897a1d95ac15061e9e4)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:47.237737 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:47.269845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:47.271364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:47.293557 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e56ba7f9cc64897a1d95ac15061e9e4', 'request_id': 'req-fc5c66cb-04e6-4744-94c2-a019f2c682d4'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:47.300056 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e56ba7f9cc64897a1d95ac15061e9e4', 'request_id': 'req-fc5c66cb-04e6-4744-94c2-a019f2c682d4'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:47.306858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e56ba7f9cc64897a1d95ac15061e9e4', 'request_id': 'req-fc5c66cb-04e6-4744-94c2-a019f2c682d4'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:47.319397 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] The token cache is being invalidate because project 5e56ba7f9cc64897a1d95ac15061e9e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:47.319620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:47.320264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:47.321981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1301/2559] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:47 2022] DELETE /identity/v3/projects/5e56ba7f9cc64897a1d95ac15061e9e4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:47.325367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:47.325367 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:47.335172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454532a-5087-44c9-9bd4-bf8d1e536674 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:47.386222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709874b-c8cb-4bc2-bc58-58d5d58288a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:47.386993 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709874b-c8cb-4bc2-bc58-58d5d58288a0 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:47.387279 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709874b-c8cb-4bc2-bc58-58d5d58288a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:47.387553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709874b-c8cb-4bc2-bc58-58d5d58288a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:47.390757 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0709874b-c8cb-4bc2-bc58-58d5d58288a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:47.390930 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c66cb-04e6-4744-94c2-a019f2c682d4 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:47.401840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacfab1-42bb-4244-93e2-8351b0e98ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:47.402514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacfab1-42bb-4244-93e2-8351b0e98ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:47.402802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacfab1-42bb-4244-93e2-8351b0e98ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:47.403077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacfab1-42bb-4244-93e2-8351b0e98ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:47.405976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1dacfab1-42bb-4244-93e2-8351b0e98ad2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:47.455426 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709874b-c8cb-4bc2-bc58-58d5d58288a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:47.456927 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709874b-c8cb-4bc2-bc58-58d5d58288a0 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:47.467731 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1259/2560] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:47.469330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f790bc-f124-47e3-a57c-a0bbacdcc65e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:47.469706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f790bc-f124-47e3-a57c-a0bbacdcc65e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:47.469893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f790bc-f124-47e3-a57c-a0bbacdcc65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:47.491104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-b6f790bc-f124-47e3-a57c-a0bbacdcc65e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:47.492939 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b6f790bc-f124-47e3-a57c-a0bbacdcc65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:47.494631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dacfab1-42bb-4244-93e2-8351b0e98ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:47.496486 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dacfab1-42bb-4244-93e2-8351b0e98ad2 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:47.505727 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1260/2561] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:47.515000 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b929b-4d33-41d4-ad87-873c59f87113 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:47.515288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b929b-4d33-41d4-ad87-873c59f87113 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:47.516380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b929b-4d33-41d4-ad87-873c59f87113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:47.519505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1302/2562] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:47.551381 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-cd2b929b-4d33-41d4-ad87-873c59f87113 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:47.553025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-cd2b929b-4d33-41d4-ad87-873c59f87113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:47.566245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1261/2563] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:47.570324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacfab1-42bb-4244-93e2-8351b0e98ad2 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:47.590912 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacfab1-42bb-4244-93e2-8351b0e98ad2 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:47.597976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691c58c-f794-4762-9cc4-aab81c59db48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:47.598873 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691c58c-f794-4762-9cc4-aab81c59db48 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:47.599373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691c58c-f794-4762-9cc4-aab81c59db48 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:47.599815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691c58c-f794-4762-9cc4-aab81c59db48 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:47.601195 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691c58c-f794-4762-9cc4-aab81c59db48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:33:47.601768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691c58c-f794-4762-9cc4-aab81c59db48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:47.603847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691c58c-f794-4762-9cc4-aab81c59db48 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:47.621229 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1303/2564] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:33:47.627062 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b929b-4d33-41d4-ad87-873c59f87113 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:47.637725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b929b-4d33-41d4-ad87-873c59f87113 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:47.642875 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8efb4-d5be-4aef-8eea-f67f3d2b1ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:47.643965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8efb4-d5be-4aef-8eea-f67f3d2b1ade admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:47.644340 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8efb4-d5be-4aef-8eea-f67f3d2b1ade admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:47.644696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8efb4-d5be-4aef-8eea-f67f3d2b1ade admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:47.645387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc8efb4-d5be-4aef-8eea-f67f3d2b1ade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:47.647466 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc8efb4-d5be-4aef-8eea-f67f3d2b1ade admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:47.687441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1262/2565] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:47 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:47.688242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691c58c-f794-4762-9cc4-aab81c59db48 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:47.699612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691c58c-f794-4762-9cc4-aab81c59db48 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:47.799937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1155295-5de0-458c-aa24-15247eb7d305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:47.800635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1155295-5de0-458c-aa24-15247eb7d305 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:47.800924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1155295-5de0-458c-aa24-15247eb7d305 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:47.801204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1155295-5de0-458c-aa24-15247eb7d305 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:47.801909 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1155295-5de0-458c-aa24-15247eb7d305 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:47.803839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1155295-5de0-458c-aa24-15247eb7d305 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:47.828296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8efb4-d5be-4aef-8eea-f67f3d2b1ade admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:47.828784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1304/2566] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:47 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:47.838926 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc8efb4-d5be-4aef-8eea-f67f3d2b1ade admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:47.845114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d41719-a960-4400-b350-65613915a68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:47.846275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d41719-a960-4400-b350-65613915a68f admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:47.846703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d41719-a960-4400-b350-65613915a68f admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:47.847127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d41719-a960-4400-b350-65613915a68f admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:47.848114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d41719-a960-4400-b350-65613915a68f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:47.849935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d41719-a960-4400-b350-65613915a68f admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:47.866641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1155295-5de0-458c-aa24-15247eb7d305 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:47.875676 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1263/2567] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:47.877607 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1155295-5de0-458c-aa24-15247eb7d305 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:47.882398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02cd1a-f3e7-4ca3-82c6-f7b17ca144f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:47.883065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02cd1a-f3e7-4ca3-82c6-f7b17ca144f1 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:47.883350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02cd1a-f3e7-4ca3-82c6-f7b17ca144f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:47.883629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02cd1a-f3e7-4ca3-82c6-f7b17ca144f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:47.884463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02cd1a-f3e7-4ca3-82c6-f7b17ca144f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:47.886100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02cd1a-f3e7-4ca3-82c6-f7b17ca144f1 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:47.903808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1305/2568] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:33:47.904939 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d41719-a960-4400-b350-65613915a68f admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:47.916353 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d41719-a960-4400-b350-65613915a68f admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:47.922233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7419b9-d651-4e6b-bd92-5dc9c410557a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:47.923433 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7419b9-d651-4e6b-bd92-5dc9c410557a admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:47.923686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7419b9-d651-4e6b-bd92-5dc9c410557a admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:47.924004 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7419b9-d651-4e6b-bd92-5dc9c410557a admin admin] PATH_INFO: `/v3/projects/14261b85a52145e9a18e244a45778f28/users/71db764e47c645dba306edb2da28d152/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:47.943326 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7419b9-d651-4e6b-bd92-5dc9c410557a admin admin] RBAC: Authorizing `identity:create_grant(project_id=14261b85a52145e9a18e244a45778f28, user_id=71db764e47c645dba306edb2da28d152, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:47.945991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7419b9-d651-4e6b-bd92-5dc9c410557a admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:47.964114 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1264/2569] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:47 2022] PUT /identity/v3/projects/14261b85a52145e9a18e244a45778f28/users/71db764e47c645dba306edb2da28d152/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:47.974742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71187cc8-7bb4-4453-a753-41aa874ed395 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:47.975042 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71187cc8-7bb4-4453-a753-41aa874ed395 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:47.975335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71187cc8-7bb4-4453-a753-41aa874ed395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:48.009673 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-71187cc8-7bb4-4453-a753-41aa874ed395 None None] MFA Rules not processed for user `71db764e47c645dba306edb2da28d152`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:48.011430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-71187cc8-7bb4-4453-a753-41aa874ed395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:48.079394 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1306/2570] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:48.083727 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7419b9-d651-4e6b-bd92-5dc9c410557a admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:48.143698 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7419b9-d651-4e6b-bd92-5dc9c410557a admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:48.148258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3187bc-7971-4006-8111-3bca6d021997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:48.148916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3187bc-7971-4006-8111-3bca6d021997 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:48.149199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3187bc-7971-4006-8111-3bca6d021997 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:48.149475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3187bc-7971-4006-8111-3bca6d021997 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:48.150158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3187bc-7971-4006-8111-3bca6d021997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:48.152062 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3187bc-7971-4006-8111-3bca6d021997 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:48.179195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1265/2571] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:48 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:48.183479 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71187cc8-7bb4-4453-a753-41aa874ed395 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:48.194897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71187cc8-7bb4-4453-a753-41aa874ed395 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:48.254212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cc0e4-dff4-4d8c-81f5-74ed6e6dbf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:48.254898 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cc0e4-dff4-4d8c-81f5-74ed6e6dbf31 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:48.255344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cc0e4-dff4-4d8c-81f5-74ed6e6dbf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:48.255619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cc0e4-dff4-4d8c-81f5-74ed6e6dbf31 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:48.256376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cc0e4-dff4-4d8c-81f5-74ed6e6dbf31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:48.258346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cc0e4-dff4-4d8c-81f5-74ed6e6dbf31 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:48.281234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1307/2572] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:48 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:48.285737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3187bc-7971-4006-8111-3bca6d021997 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:48.298217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3187bc-7971-4006-8111-3bca6d021997 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:48.304280 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace80d3-e872-4a2d-85ab-bcc965677132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:48.304954 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace80d3-e872-4a2d-85ab-bcc965677132 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:48.305239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace80d3-e872-4a2d-85ab-bcc965677132 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:48.305515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace80d3-e872-4a2d-85ab-bcc965677132 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:48.306371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace80d3-e872-4a2d-85ab-bcc965677132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:48.308656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace80d3-e872-4a2d-85ab-bcc965677132 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:48.323065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1266/2573] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:33:48.328171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cc0e4-dff4-4d8c-81f5-74ed6e6dbf31 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:48.341278 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cc0e4-dff4-4d8c-81f5-74ed6e6dbf31 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:48.348193 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75176397-7b6a-4fd7-96ab-c547d40fb0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:48.349095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75176397-7b6a-4fd7-96ab-c547d40fb0ec admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:48.349382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75176397-7b6a-4fd7-96ab-c547d40fb0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:48.349660 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75176397-7b6a-4fd7-96ab-c547d40fb0ec admin admin] PATH_INFO: `/v3/projects/0999d12683724a1c9710bf353a12f4ea/users/8ceb127fec0b42988db1ad9b233da8cc/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:48.368149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75176397-7b6a-4fd7-96ab-c547d40fb0ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=0999d12683724a1c9710bf353a12f4ea, user_id=8ceb127fec0b42988db1ad9b233da8cc, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:48.370781 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75176397-7b6a-4fd7-96ab-c547d40fb0ec admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:48.386309 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1308/2574] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:48 2022] PUT /identity/v3/projects/0999d12683724a1c9710bf353a12f4ea/users/8ceb127fec0b42988db1ad9b233da8cc/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:48.390613 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace80d3-e872-4a2d-85ab-bcc965677132 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:48.400599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace80d3-e872-4a2d-85ab-bcc965677132 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:48.463874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffc06e-021b-4c46-aa34-22849c739fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:48.465198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffc06e-021b-4c46-aa34-22849c739fcf admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:48.465481 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffc06e-021b-4c46-aa34-22849c739fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:48.465755 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffc06e-021b-4c46-aa34-22849c739fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:48.466520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffc06e-021b-4c46-aa34-22849c739fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:48.468534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffc06e-021b-4c46-aa34-22849c739fcf admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:48.479211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1267/2575] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:48.484895 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75176397-7b6a-4fd7-96ab-c547d40fb0ec admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:48.495973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75176397-7b6a-4fd7-96ab-c547d40fb0ec admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:48.502091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a736fe-d547-4ab5-b0ab-78b688712d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:48.503633 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a736fe-d547-4ab5-b0ab-78b688712d12 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:48.504186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a736fe-d547-4ab5-b0ab-78b688712d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:48.504539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a736fe-d547-4ab5-b0ab-78b688712d12 admin admin] PATH_INFO: `/v3/projects/0999d12683724a1c9710bf353a12f4ea/users/8ceb127fec0b42988db1ad9b233da8cc/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:48.506634 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a736fe-d547-4ab5-b0ab-78b688712d12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0999d12683724a1c9710bf353a12f4ea, user_id=8ceb127fec0b42988db1ad9b233da8cc, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:48.509380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a736fe-d547-4ab5-b0ab-78b688712d12 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:48.520446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1309/2576] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:48 2022] PUT /identity/v3/projects/0999d12683724a1c9710bf353a12f4ea/users/8ceb127fec0b42988db1ad9b233da8cc/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:48.531270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080de75-136f-4b68-b4ed-bcf44843b99a None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:48.531626 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080de75-136f-4b68-b4ed-bcf44843b99a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:48.531928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080de75-136f-4b68-b4ed-bcf44843b99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:48.564346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-1080de75-136f-4b68-b4ed-bcf44843b99a None None] MFA Rules not processed for user `8ceb127fec0b42988db1ad9b233da8cc`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:48.565822 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1080de75-136f-4b68-b4ed-bcf44843b99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:48.658746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1268/2577] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:48.672287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a736fe-d547-4ab5-b0ab-78b688712d12 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:48.674316 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a8a736fe-d547-4ab5-b0ab-78b688712d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:48.780096 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a736fe-d547-4ab5-b0ab-78b688712d12 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:48.786224 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10cee6-070c-4849-aaeb-8b96a58fdd90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:48.787332 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10cee6-070c-4849-aaeb-8b96a58fdd90 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:48.787771 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10cee6-070c-4849-aaeb-8b96a58fdd90 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:48.788233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10cee6-070c-4849-aaeb-8b96a58fdd90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:48.800666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10cee6-070c-4849-aaeb-8b96a58fdd90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:48.802928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10cee6-070c-4849-aaeb-8b96a58fdd90 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:48.826302 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1310/2578] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:48.939129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080de75-136f-4b68-b4ed-bcf44843b99a None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:49.003345 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080de75-136f-4b68-b4ed-bcf44843b99a None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:49.007776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:49.008689 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:49.008963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:49.009230 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] PATH_INFO: `/v3/users/71db764e47c645dba306edb2da28d152` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:49.010428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71db764e47c645dba306edb2da28d152)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:49.013433 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:49.070191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71db764e47c645dba306edb2da28d152'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:49.071145 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71db764e47c645dba306edb2da28d152'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:49.071758 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71db764e47c645dba306edb2da28d152'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:49.079623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71db764e47c645dba306edb2da28d152'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:49.086757 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71db764e47c645dba306edb2da28d152'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:49.103627 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1269/2579] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:48 2022] DELETE /identity/v3/users/71db764e47c645dba306edb2da28d152 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:49.109054 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9c32c-f9a3-45b6-8e2f-d785b1cdaf31 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:49.109338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9c32c-f9a3-45b6-8e2f-d785b1cdaf31 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:49.109609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9c32c-f9a3-45b6-8e2f-d785b1cdaf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:49.127527 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-cee9c32c-f9a3-45b6-8e2f-d785b1cdaf31 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:49.128751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cee9c32c-f9a3-45b6-8e2f-d785b1cdaf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:49.187629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1311/2580] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:49.198698 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:49.200257 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:49.256050 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663f270-e5a9-4924-af7b-7962deb97db2 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:49.259736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6b557-6edb-4979-8dbf-f39ae0a347df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:49.260418 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6b557-6edb-4979-8dbf-f39ae0a347df service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:49.260705 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6b557-6edb-4979-8dbf-f39ae0a347df service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:49.260986 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6b557-6edb-4979-8dbf-f39ae0a347df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:49.270322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6b557-6edb-4979-8dbf-f39ae0a347df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:49.271772 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6b557-6edb-4979-8dbf-f39ae0a347df service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:49.281867 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1270/2581] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:49.684882 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9c32c-f9a3-45b6-8e2f-d785b1cdaf31 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:49.686493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cee9c32c-f9a3-45b6-8e2f-d785b1cdaf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:49.712166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9c32c-f9a3-45b6-8e2f-d785b1cdaf31 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:49.732470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:49.733517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:49.734443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:49.734731 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] PATH_INFO: `/v3/projects/14261b85a52145e9a18e244a45778f28` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:49.736067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14261b85a52145e9a18e244a45778f28)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:49.738003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:49.799353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14261b85a52145e9a18e244a45778f28', 'request_id': 'req-d0d662d9-70f7-453f-bb86-179f0cf44286'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:49.806117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14261b85a52145e9a18e244a45778f28', 'request_id': 'req-d0d662d9-70f7-453f-bb86-179f0cf44286'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:49.815049 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14261b85a52145e9a18e244a45778f28', 'request_id': 'req-d0d662d9-70f7-453f-bb86-179f0cf44286'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:49.833227 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] The token cache is being invalidate because project 14261b85a52145e9a18e244a45778f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:49.833525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:49.834885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:49.840328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1312/2582] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:49 2022] DELETE /identity/v3/projects/14261b85a52145e9a18e244a45778f28 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:49.842158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6b557-6edb-4979-8dbf-f39ae0a347df service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:49.852347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6b557-6edb-4979-8dbf-f39ae0a347df service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:49.930812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e37da5-a15e-4fb7-8c0d-7a548dc471d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:49.931845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e37da5-a15e-4fb7-8c0d-7a548dc471d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:49.932171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e37da5-a15e-4fb7-8c0d-7a548dc471d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:49.932480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e37da5-a15e-4fb7-8c0d-7a548dc471d8 admin admin] PATH_INFO: `/v3/users/8ceb127fec0b42988db1ad9b233da8cc` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:49.933912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e37da5-a15e-4fb7-8c0d-7a548dc471d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ceb127fec0b42988db1ad9b233da8cc)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:49.936087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e37da5-a15e-4fb7-8c0d-7a548dc471d8 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:49.988635 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-19e37da5-a15e-4fb7-8c0d-7a548dc471d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ceb127fec0b42988db1ad9b233da8cc'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:49.989638 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-19e37da5-a15e-4fb7-8c0d-7a548dc471d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ceb127fec0b42988db1ad9b233da8cc'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:49.990250 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-19e37da5-a15e-4fb7-8c0d-7a548dc471d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ceb127fec0b42988db1ad9b233da8cc'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:49.998444 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-19e37da5-a15e-4fb7-8c0d-7a548dc471d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ceb127fec0b42988db1ad9b233da8cc'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:50.005870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-19e37da5-a15e-4fb7-8c0d-7a548dc471d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ceb127fec0b42988db1ad9b233da8cc'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:50.024131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1271/2583] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:49 2022] DELETE /identity/v3/users/8ceb127fec0b42988db1ad9b233da8cc => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:50.349744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:50.351246 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:50.374315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d662d9-70f7-453f-bb86-179f0cf44286 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:50.434325 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85842c31-ab9a-4cf2-a3cc-e318c988d829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:50.436141 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85842c31-ab9a-4cf2-a3cc-e318c988d829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:50.436421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85842c31-ab9a-4cf2-a3cc-e318c988d829 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:50.436695 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85842c31-ab9a-4cf2-a3cc-e318c988d829 admin admin] PATH_INFO: `/v3/projects/0999d12683724a1c9710bf353a12f4ea` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:50.437884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85842c31-ab9a-4cf2-a3cc-e318c988d829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0999d12683724a1c9710bf353a12f4ea)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:50.439805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85842c31-ab9a-4cf2-a3cc-e318c988d829 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:50.494442 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-85842c31-ab9a-4cf2-a3cc-e318c988d829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0999d12683724a1c9710bf353a12f4ea', 'request_id': 'req-85842c31-ab9a-4cf2-a3cc-e318c988d829'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:50.501111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-85842c31-ab9a-4cf2-a3cc-e318c988d829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0999d12683724a1c9710bf353a12f4ea', 'request_id': 'req-85842c31-ab9a-4cf2-a3cc-e318c988d829'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:50.510380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-85842c31-ab9a-4cf2-a3cc-e318c988d829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0999d12683724a1c9710bf353a12f4ea', 'request_id': 'req-85842c31-ab9a-4cf2-a3cc-e318c988d829'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:50.524078 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-85842c31-ab9a-4cf2-a3cc-e318c988d829 admin admin] The token cache is being invalidate because project 0999d12683724a1c9710bf353a12f4ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:50.524362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-85842c31-ab9a-4cf2-a3cc-e318c988d829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:50.525216 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-85842c31-ab9a-4cf2-a3cc-e318c988d829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:50.528246 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1313/2584] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:50 2022] DELETE /identity/v3/projects/0999d12683724a1c9710bf353a12f4ea => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:50.532475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd1f48-ab0e-464c-834b-547e70257d1d None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:50.532768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd1f48-ab0e-464c-834b-547e70257d1d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:50.533052 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd1f48-ab0e-464c-834b-547e70257d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:50.564296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-1ccd1f48-ab0e-464c-834b-547e70257d1d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:50.565504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1ccd1f48-ab0e-464c-834b-547e70257d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:50.638550 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1272/2585] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:50.647596 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd614fe7-d9b7-4561-8106-09ab5358cdac None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:50.647982 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd614fe7-d9b7-4561-8106-09ab5358cdac None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:50.648519 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd614fe7-d9b7-4561-8106-09ab5358cdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:50.677107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-cd614fe7-d9b7-4561-8106-09ab5358cdac None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:50.684341 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cd614fe7-d9b7-4561-8106-09ab5358cdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:50.711107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1314/2586] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:50.711328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd1f48-ab0e-464c-834b-547e70257d1d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:50.723380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd1f48-ab0e-464c-834b-547e70257d1d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:50.731019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f772b-a665-4433-befb-a7e54cf3a094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:50.731717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f772b-a665-4433-befb-a7e54cf3a094 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:50.733124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f772b-a665-4433-befb-a7e54cf3a094 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:50.733413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f772b-a665-4433-befb-a7e54cf3a094 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:50.734159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f772b-a665-4433-befb-a7e54cf3a094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:33:50.734469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f772b-a665-4433-befb-a7e54cf3a094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:50.736272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f772b-a665-4433-befb-a7e54cf3a094 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:50.766425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1273/2587] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:33:50.771004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd614fe7-d9b7-4561-8106-09ab5358cdac None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:50.784421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd614fe7-d9b7-4561-8106-09ab5358cdac None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:50.789455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852f12c-b733-4044-8c41-53087092ced0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:50.790353 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852f12c-b733-4044-8c41-53087092ced0 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:50.790632 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852f12c-b733-4044-8c41-53087092ced0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:50.791123 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852f12c-b733-4044-8c41-53087092ced0 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:50.792012 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852f12c-b733-4044-8c41-53087092ced0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:50.793963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852f12c-b733-4044-8c41-53087092ced0 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:50.825148 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1315/2588] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:50 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:50.830620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f772b-a665-4433-befb-a7e54cf3a094 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:50.844789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f772b-a665-4433-befb-a7e54cf3a094 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:50.936936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32655205-942a-432e-bba5-e197692c8d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:50.937845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32655205-942a-432e-bba5-e197692c8d52 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:50.938230 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32655205-942a-432e-bba5-e197692c8d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:50.938796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32655205-942a-432e-bba5-e197692c8d52 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:50.940541 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32655205-942a-432e-bba5-e197692c8d52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:50.943108 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32655205-942a-432e-bba5-e197692c8d52 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:50.979909 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1274/2589] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:50 2022] POST /identity/v3/users => generated 294 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:50.984665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852f12c-b733-4044-8c41-53087092ced0 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:50.996699 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852f12c-b733-4044-8c41-53087092ced0 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:51.004117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4497e0-ad83-427d-a8ec-1208cd596d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:51.004942 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4497e0-ad83-427d-a8ec-1208cd596d87 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:51.005231 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4497e0-ad83-427d-a8ec-1208cd596d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:51.005710 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4497e0-ad83-427d-a8ec-1208cd596d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:51.006535 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4497e0-ad83-427d-a8ec-1208cd596d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:51.008398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4497e0-ad83-427d-a8ec-1208cd596d87 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:51.035148 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1316/2590] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:51.038904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32655205-942a-432e-bba5-e197692c8d52 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:51.062721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32655205-942a-432e-bba5-e197692c8d52 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:51.069514 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec16a9-b1da-49c5-8688-c90ed36dcf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:51.070242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec16a9-b1da-49c5-8688-c90ed36dcf25 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:51.070903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec16a9-b1da-49c5-8688-c90ed36dcf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:51.071358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec16a9-b1da-49c5-8688-c90ed36dcf25 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:51.072447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec16a9-b1da-49c5-8688-c90ed36dcf25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:51.076518 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec16a9-b1da-49c5-8688-c90ed36dcf25 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:51.093043 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1275/2591] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:51.097452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4497e0-ad83-427d-a8ec-1208cd596d87 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:51.119323 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4497e0-ad83-427d-a8ec-1208cd596d87 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:51.131305 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd818c-50c1-4d34-9e8d-7afcf14bc40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:51.132411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd818c-50c1-4d34-9e8d-7afcf14bc40e admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:51.132789 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd818c-50c1-4d34-9e8d-7afcf14bc40e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:51.133180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd818c-50c1-4d34-9e8d-7afcf14bc40e admin admin] PATH_INFO: `/v3/projects/2379c1ea639149f0b4a421a39faa18be/users/efe5a4196da141a28bcc2b033c2bf7a7/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:51.152125 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd818c-50c1-4d34-9e8d-7afcf14bc40e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2379c1ea639149f0b4a421a39faa18be, user_id=efe5a4196da141a28bcc2b033c2bf7a7, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:51.155931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd818c-50c1-4d34-9e8d-7afcf14bc40e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:51.197215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1317/2592] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:51 2022] PUT /identity/v3/projects/2379c1ea639149f0b4a421a39faa18be/users/efe5a4196da141a28bcc2b033c2bf7a7/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:51.208826 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e74db-a8ef-485a-a98d-b11420b78f61 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:51.209333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e74db-a8ef-485a-a98d-b11420b78f61 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:51.209761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e74db-a8ef-485a-a98d-b11420b78f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:51.271839 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-0a9e74db-a8ef-485a-a98d-b11420b78f61 None None] MFA Rules not processed for user `efe5a4196da141a28bcc2b033c2bf7a7`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:51.277858 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0a9e74db-a8ef-485a-a98d-b11420b78f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:51.298547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd818c-50c1-4d34-9e8d-7afcf14bc40e admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:51.300586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-bbdd818c-50c1-4d34-9e8d-7afcf14bc40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:51.414858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd818c-50c1-4d34-9e8d-7afcf14bc40e admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:51.430717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:51.431834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:51.434364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:51.434733 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] PATH_INFO: `/v3/users/30970d0f6a70463ca566586955310ea6` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:51.438149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] RBAC: Authorizing `identity:delete_user(user_id=30970d0f6a70463ca566586955310ea6)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:51.442088 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:51.448801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1276/2593] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:51.453579 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e74db-a8ef-485a-a98d-b11420b78f61 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:51.465397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e74db-a8ef-485a-a98d-b11420b78f61 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:51.472210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b713d-5dd1-4941-82c4-343e41e6ea94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:51.473376 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b713d-5dd1-4941-82c4-343e41e6ea94 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:51.473783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b713d-5dd1-4941-82c4-343e41e6ea94 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:51.474243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b713d-5dd1-4941-82c4-343e41e6ea94 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:51.475128 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b713d-5dd1-4941-82c4-343e41e6ea94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:51.478553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b713d-5dd1-4941-82c4-343e41e6ea94 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:51.522785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1277/2594] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:51 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:51.526133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b713d-5dd1-4941-82c4-343e41e6ea94 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:51.542357 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30970d0f6a70463ca566586955310ea6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:51.544061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30970d0f6a70463ca566586955310ea6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:51.557439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30970d0f6a70463ca566586955310ea6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:51.576155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30970d0f6a70463ca566586955310ea6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:51.578302 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30970d0f6a70463ca566586955310ea6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:51.619144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1318/2595] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:51 2022] DELETE /identity/v3/users/30970d0f6a70463ca566586955310ea6 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:51.620232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:51.621849 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:51.656090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b713d-5dd1-4941-82c4-343e41e6ea94 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:51.705793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fd5d5-97ed-43e1-8a34-8e878ecd264e admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:51.710657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:51.711538 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:51.711822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:51.712120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] PATH_INFO: `/v3/users/c56a403eeee84720ba765e2d4367df74` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:51.713328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c56a403eeee84720ba765e2d4367df74)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:51.715496 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:51.719693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3cb9d-a917-411a-9dba-3ae443d935bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:51.720429 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3cb9d-a917-411a-9dba-3ae443d935bb admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:51.720713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3cb9d-a917-411a-9dba-3ae443d935bb admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:51.720985 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3cb9d-a917-411a-9dba-3ae443d935bb admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:51.721672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3cb9d-a917-411a-9dba-3ae443d935bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:51.723592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3cb9d-a917-411a-9dba-3ae443d935bb admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:51.750181 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1278/2596] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:51 2022] POST /identity/v3/users => generated 300 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:51.752012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191c1b0-7a78-4184-a178-e2dd1e0b838d None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:51.752417 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191c1b0-7a78-4184-a178-e2dd1e0b838d None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:51.752703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191c1b0-7a78-4184-a178-e2dd1e0b838d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:51.767084 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c56a403eeee84720ba765e2d4367df74'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:51.767800 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c56a403eeee84720ba765e2d4367df74'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:51.774695 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c56a403eeee84720ba765e2d4367df74'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:51.775582 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-4191c1b0-7a78-4184-a178-e2dd1e0b838d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:51.776809 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-4191c1b0-7a78-4184-a178-e2dd1e0b838d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:51.782229 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c56a403eeee84720ba765e2d4367df74'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:51.782854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c56a403eeee84720ba765e2d4367df74'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:51.785162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1279/2597] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:51.785933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191c1b0-7a78-4184-a178-e2dd1e0b838d None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:51.787124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-4191c1b0-7a78-4184-a178-e2dd1e0b838d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:51.799362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191c1b0-7a78-4184-a178-e2dd1e0b838d None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:51.801409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1319/2598] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:51 2022] DELETE /identity/v3/users/c56a403eeee84720ba765e2d4367df74 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:51.802042 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:51.803315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:51.862082 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340044b3-b949-4f78-abe5-95daead2e1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:51.862758 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340044b3-b949-4f78-abe5-95daead2e1bf admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:51.863042 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340044b3-b949-4f78-abe5-95daead2e1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:51.863312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340044b3-b949-4f78-abe5-95daead2e1bf admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:51.863795 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee48a8-854c-43a8-bd1a-3ac793e6f28d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:51.864180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340044b3-b949-4f78-abe5-95daead2e1bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:51.865645 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340044b3-b949-4f78-abe5-95daead2e1bf admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:51.867527 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dded7e3-7eb4-42e6-98cd-65217ec5ee71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:51.868206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dded7e3-7eb4-42e6-98cd-65217ec5ee71 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:51.868483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dded7e3-7eb4-42e6-98cd-65217ec5ee71 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:51.868756 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dded7e3-7eb4-42e6-98cd-65217ec5ee71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:51.871040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-0dded7e3-7eb4-42e6-98cd-65217ec5ee71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:51.876941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1280/2599] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:51.880997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340044b3-b949-4f78-abe5-95daead2e1bf admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:51.883725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dded7e3-7eb4-42e6-98cd-65217ec5ee71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:51.884776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-340044b3-b949-4f78-abe5-95daead2e1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:51.885190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dded7e3-7eb4-42e6-98cd-65217ec5ee71 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:51.898266 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1320/2600] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:51.911208 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340044b3-b949-4f78-abe5-95daead2e1bf admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:51.920041 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b5c3b-8653-498a-8a86-dd6c2f357df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:51.922095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b5c3b-8653-498a-8a86-dd6c2f357df7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:51.922591 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b5c3b-8653-498a-8a86-dd6c2f357df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:51.923380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b5c3b-8653-498a-8a86-dd6c2f357df7 admin admin] PATH_INFO: `/v3/projects/a7de06aa34ae489e9671f220b5ee13da/users/96beefe6b93540e7a5fff98d47af8311/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:51.942873 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b5c3b-8653-498a-8a86-dd6c2f357df7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7de06aa34ae489e9671f220b5ee13da, user_id=96beefe6b93540e7a5fff98d47af8311, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:51.946699 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b5c3b-8653-498a-8a86-dd6c2f357df7 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:51.961179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1281/2601] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:51 2022] PUT /identity/v3/projects/a7de06aa34ae489e9671f220b5ee13da/users/96beefe6b93540e7a5fff98d47af8311/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:51.965804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dded7e3-7eb4-42e6-98cd-65217ec5ee71 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:51.979777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dded7e3-7eb4-42e6-98cd-65217ec5ee71 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:51.996008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b5c3b-8653-498a-8a86-dd6c2f357df7 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:51.997989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-9b0b5c3b-8653-498a-8a86-dd6c2f357df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:52.087265 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b5c3b-8653-498a-8a86-dd6c2f357df7 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:52.094941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8fa13-530e-464f-903a-65f29691cb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:52.095636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8fa13-530e-464f-903a-65f29691cb8b admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.095999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8fa13-530e-464f-903a-65f29691cb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.096179 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8fa13-530e-464f-903a-65f29691cb8b admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.096413 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28821fab-b08c-4272-ae8d-38030c2687ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:52.096951 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8fa13-530e-464f-903a-65f29691cb8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:52.098086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28821fab-b08c-4272-ae8d-38030c2687ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.098495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8fa13-530e-464f-903a-65f29691cb8b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:52.098692 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28821fab-b08c-4272-ae8d-38030c2687ca admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.099140 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28821fab-b08c-4272-ae8d-38030c2687ca admin admin] PATH_INFO: `/v3/projects/9c83b0a8b88f452f9e12d077e8ac28c8` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.101362 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28821fab-b08c-4272-ae8d-38030c2687ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c83b0a8b88f452f9e12d077e8ac28c8)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:52.103295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28821fab-b08c-4272-ae8d-38030c2687ca admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:52.123516 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1321/2602] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:52.127466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8fa13-530e-464f-903a-65f29691cb8b admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:52.138761 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8fa13-530e-464f-903a-65f29691cb8b admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:52.145175 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f25e35-5651-4988-b50e-1c3896e97ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:52.146027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f25e35-5651-4988-b50e-1c3896e97ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.146356 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f25e35-5651-4988-b50e-1c3896e97ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.146663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f25e35-5651-4988-b50e-1c3896e97ac7 admin admin] PATH_INFO: `/v3/projects/a7de06aa34ae489e9671f220b5ee13da/users/96beefe6b93540e7a5fff98d47af8311/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.149230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f25e35-5651-4988-b50e-1c3896e97ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7de06aa34ae489e9671f220b5ee13da, user_id=96beefe6b93540e7a5fff98d47af8311, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:52.152144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f25e35-5651-4988-b50e-1c3896e97ac7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:52.164644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1322/2603] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:52 2022] PUT /identity/v3/projects/a7de06aa34ae489e9671f220b5ee13da/users/96beefe6b93540e7a5fff98d47af8311/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:52.170861 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-28821fab-b08c-4272-ae8d-38030c2687ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c83b0a8b88f452f9e12d077e8ac28c8', 'request_id': 'req-28821fab-b08c-4272-ae8d-38030c2687ca'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.174820 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb05e11-d1d1-455c-af3f-bb81c9480f28 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.175110 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb05e11-d1d1-455c-af3f-bb81c9480f28 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.175390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb05e11-d1d1-455c-af3f-bb81c9480f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.178085 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-28821fab-b08c-4272-ae8d-38030c2687ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c83b0a8b88f452f9e12d077e8ac28c8', 'request_id': 'req-28821fab-b08c-4272-ae8d-38030c2687ca'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.187046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-28821fab-b08c-4272-ae8d-38030c2687ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c83b0a8b88f452f9e12d077e8ac28c8', 'request_id': 'req-28821fab-b08c-4272-ae8d-38030c2687ca'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.203376 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-28821fab-b08c-4272-ae8d-38030c2687ca admin admin] The token cache is being invalidate because project 9c83b0a8b88f452f9e12d077e8ac28c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:52.203691 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-28821fab-b08c-4272-ae8d-38030c2687ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.204351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-28821fab-b08c-4272-ae8d-38030c2687ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.206031 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1282/2604] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:51 2022] DELETE /identity/v3/projects/9c83b0a8b88f452f9e12d077e8ac28c8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:52.208015 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-cfb05e11-d1d1-455c-af3f-bb81c9480f28 None None] MFA Rules not processed for user `96beefe6b93540e7a5fff98d47af8311`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:52.209700 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cfb05e11-d1d1-455c-af3f-bb81c9480f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:52.210792 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9decb38-6504-4d3e-82f1-e8d5288b3547 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.211172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9decb38-6504-4d3e-82f1-e8d5288b3547 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.211378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9decb38-6504-4d3e-82f1-e8d5288b3547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.232414 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-e9decb38-6504-4d3e-82f1-e8d5288b3547 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:52.233606 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e9decb38-6504-4d3e-82f1-e8d5288b3547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:52.292379 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1323/2605] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:52.296976 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb05e11-d1d1-455c-af3f-bb81c9480f28 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:52.298899 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cfb05e11-d1d1-455c-af3f-bb81c9480f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:52.308686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1283/2606] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:52.315801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9decb38-6504-4d3e-82f1-e8d5288b3547 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:52.315801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e9decb38-6504-4d3e-82f1-e8d5288b3547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:52.369645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb05e11-d1d1-455c-af3f-bb81c9480f28 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:52.371446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9decb38-6504-4d3e-82f1-e8d5288b3547 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:52.375010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:52.375814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74dca0-592e-4963-a9c3-f86df90f9706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:52.375979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.376273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.376496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74dca0-592e-4963-a9c3-f86df90f9706 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.376618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] PATH_INFO: `/v3/projects/e30fec6f567c4058bf9d445e96cecb29` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.376781 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74dca0-592e-4963-a9c3-f86df90f9706 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.377055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74dca0-592e-4963-a9c3-f86df90f9706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.378491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e30fec6f567c4058bf9d445e96cecb29)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:52.380455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:52.385784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74dca0-592e-4963-a9c3-f86df90f9706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:52.387177 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74dca0-592e-4963-a9c3-f86df90f9706 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:52.397131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1284/2607] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:52.398350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070bce0-a995-409f-b93d-8cab6987cf78 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.398632 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070bce0-a995-409f-b93d-8cab6987cf78 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.398904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070bce0-a995-409f-b93d-8cab6987cf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.415949 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-0070bce0-a995-409f-b93d-8cab6987cf78 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:52.417060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0070bce0-a995-409f-b93d-8cab6987cf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:52.430779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e30fec6f567c4058bf9d445e96cecb29', 'request_id': 'req-2e01fe99-b37f-40bd-b84d-c479b568fb74'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.437079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e30fec6f567c4058bf9d445e96cecb29', 'request_id': 'req-2e01fe99-b37f-40bd-b84d-c479b568fb74'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.445561 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e30fec6f567c4058bf9d445e96cecb29', 'request_id': 'req-2e01fe99-b37f-40bd-b84d-c479b568fb74'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.458406 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] The token cache is being invalidate because project e30fec6f567c4058bf9d445e96cecb29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:52.458614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.459230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.461201 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1324/2608] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:52 2022] DELETE /identity/v3/projects/e30fec6f567c4058bf9d445e96cecb29 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:52.469116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:52.470586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:52.497109 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1285/2609] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:52.502356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070bce0-a995-409f-b93d-8cab6987cf78 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:52.511415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070bce0-a995-409f-b93d-8cab6987cf78 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:52.517311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b243ca3-4320-4d97-8047-58344d6170bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:52.517694 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b243ca3-4320-4d97-8047-58344d6170bf admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.517978 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b243ca3-4320-4d97-8047-58344d6170bf admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.518249 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b243ca3-4320-4d97-8047-58344d6170bf admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.518953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b243ca3-4320-4d97-8047-58344d6170bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:33:52.519397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b243ca3-4320-4d97-8047-58344d6170bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:52.521133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b243ca3-4320-4d97-8047-58344d6170bf admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:52.532870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1286/2610] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:33:52.537559 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b243ca3-4320-4d97-8047-58344d6170bf admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:52.543475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01fe99-b37f-40bd-b84d-c479b568fb74 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:52.547668 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b243ca3-4320-4d97-8047-58344d6170bf admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:52.548570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:52.549448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.549873 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.550172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] PATH_INFO: `/v3/users/1a60bd63cac44fd4b5dc3c9372515bbe` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.551391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a60bd63cac44fd4b5dc3c9372515bbe)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:52.552841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78728a9-f9a9-4506-8416-da26e190f931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:52.553445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:52.553585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78728a9-f9a9-4506-8416-da26e190f931 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.553936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78728a9-f9a9-4506-8416-da26e190f931 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.554222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78728a9-f9a9-4506-8416-da26e190f931 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.554991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78728a9-f9a9-4506-8416-da26e190f931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:52.556907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78728a9-f9a9-4506-8416-da26e190f931 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:52.586049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1287/2611] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:52 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:52.590282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78728a9-f9a9-4506-8416-da26e190f931 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:52.619418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a60bd63cac44fd4b5dc3c9372515bbe'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.620927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a60bd63cac44fd4b5dc3c9372515bbe'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.630930 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a60bd63cac44fd4b5dc3c9372515bbe'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.638802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a60bd63cac44fd4b5dc3c9372515bbe'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.639566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a60bd63cac44fd4b5dc3c9372515bbe'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:52.658143 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1325/2612] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:52 2022] DELETE /identity/v3/users/1a60bd63cac44fd4b5dc3c9372515bbe => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:52.675775 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78728a9-f9a9-4506-8416-da26e190f931 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:52.752244 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74fe80-f568-4885-aa81-2c0a6a3239bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:52.753444 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74fe80-f568-4885-aa81-2c0a6a3239bb admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.754136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74fe80-f568-4885-aa81-2c0a6a3239bb admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.754136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74fe80-f568-4885-aa81-2c0a6a3239bb admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.754820 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74fe80-f568-4885-aa81-2c0a6a3239bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:52.758448 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74fe80-f568-4885-aa81-2c0a6a3239bb admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:52.788065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1288/2613] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:52 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:52.792947 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:52.794962 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:52.810670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396bae3-593c-4529-bfc8-ada65b22c820 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:52.817097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffaaa1-4012-4d32-bc2a-01977178ba94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:52.817931 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffaaa1-4012-4d32-bc2a-01977178ba94 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.818268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffaaa1-4012-4d32-bc2a-01977178ba94 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.818889 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffaaa1-4012-4d32-bc2a-01977178ba94 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.819684 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffaaa1-4012-4d32-bc2a-01977178ba94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:52.821326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffaaa1-4012-4d32-bc2a-01977178ba94 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:52.837697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1326/2614] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:33:52.839970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74fe80-f568-4885-aa81-2c0a6a3239bb admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:52.865159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74fe80-f568-4885-aa81-2c0a6a3239bb admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:52.870883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bcb8e-7f51-4d47-a009-80d8ef156035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:52.871592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bcb8e-7f51-4d47-a009-80d8ef156035 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.871982 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bcb8e-7f51-4d47-a009-80d8ef156035 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.872269 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bcb8e-7f51-4d47-a009-80d8ef156035 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.874245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229bcb8e-7f51-4d47-a009-80d8ef156035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:52.876091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229bcb8e-7f51-4d47-a009-80d8ef156035 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:52.887272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1289/2615] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:52.891270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffaaa1-4012-4d32-bc2a-01977178ba94 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:52.904652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffaaa1-4012-4d32-bc2a-01977178ba94 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:52.910629 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69deb79c-01b0-4bb1-b890-ac79cf2b97aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:52.911319 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69deb79c-01b0-4bb1-b890-ac79cf2b97aa admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.911708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69deb79c-01b0-4bb1-b890-ac79cf2b97aa admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.912168 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69deb79c-01b0-4bb1-b890-ac79cf2b97aa admin admin] PATH_INFO: `/v3/projects/5eaf003451d34d28b0795a072e96a44c/users/674f29fe1441416f9a22e96fa82d35e3/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.932171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69deb79c-01b0-4bb1-b890-ac79cf2b97aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eaf003451d34d28b0795a072e96a44c, user_id=674f29fe1441416f9a22e96fa82d35e3, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:52.933630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69deb79c-01b0-4bb1-b890-ac79cf2b97aa admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:52.951750 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1327/2616] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:52 2022] PUT /identity/v3/projects/5eaf003451d34d28b0795a072e96a44c/users/674f29fe1441416f9a22e96fa82d35e3/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:52.966225 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de84b0-a2e1-4589-85f5-41bc3c57394b None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:52.966561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de84b0-a2e1-4589-85f5-41bc3c57394b None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:52.966884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de84b0-a2e1-4589-85f5-41bc3c57394b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:52.998840 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-13de84b0-a2e1-4589-85f5-41bc3c57394b None None] MFA Rules not processed for user `674f29fe1441416f9a22e96fa82d35e3`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:52.999442 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69deb79c-01b0-4bb1-b890-ac79cf2b97aa admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:53.000083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-13de84b0-a2e1-4589-85f5-41bc3c57394b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:53.001581 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-69deb79c-01b0-4bb1-b890-ac79cf2b97aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:53.079540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69deb79c-01b0-4bb1-b890-ac79cf2b97aa admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:53.084271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:53.085140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:53.085423 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:53.085704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] PATH_INFO: `/v3/projects/c29d8ea15635473b9977f455ac5437bb` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:53.086897 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29d8ea15635473b9977f455ac5437bb)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:53.088846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:53.094553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1290/2617] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:53.095391 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de84b0-a2e1-4589-85f5-41bc3c57394b None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:53.099355 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-13de84b0-a2e1-4589-85f5-41bc3c57394b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:53.112813 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de84b0-a2e1-4589-85f5-41bc3c57394b None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:53.117631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:53.119801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:53.119801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:53.119801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] PATH_INFO: `/v3/users/efe5a4196da141a28bcc2b033c2bf7a7` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:53.124851 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efe5a4196da141a28bcc2b033c2bf7a7)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:53.128505 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:53.146121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29d8ea15635473b9977f455ac5437bb', 'request_id': 'req-0de8b0d6-8f8c-4fac-b1cd-563888e76944'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.152843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29d8ea15635473b9977f455ac5437bb', 'request_id': 'req-0de8b0d6-8f8c-4fac-b1cd-563888e76944'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.161483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29d8ea15635473b9977f455ac5437bb', 'request_id': 'req-0de8b0d6-8f8c-4fac-b1cd-563888e76944'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.179534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] The token cache is being invalidate because project c29d8ea15635473b9977f455ac5437bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:53.179534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.179534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.179534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1328/2618] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:52 2022] DELETE /identity/v3/projects/c29d8ea15635473b9977f455ac5437bb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:33:53.179534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:53.181263 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:53.183094 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efe5a4196da141a28bcc2b033c2bf7a7'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.183741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efe5a4196da141a28bcc2b033c2bf7a7'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.184380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efe5a4196da141a28bcc2b033c2bf7a7'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.193746 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efe5a4196da141a28bcc2b033c2bf7a7'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.200537 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efe5a4196da141a28bcc2b033c2bf7a7'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.218332 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1291/2619] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:53 2022] DELETE /identity/v3/users/efe5a4196da141a28bcc2b033c2bf7a7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:53.219126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:53.220472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:53.251826 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8b0d6-8f8c-4fac-b1cd-563888e76944 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:53.287539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9485a-4ada-486b-9316-e4944a7b5fb6 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:53.293320 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:53.294181 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:53.294501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:53.294780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] PATH_INFO: `/v3/users/ecedc4e5ddb643e0b0852dc72cfdcc8c` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:53.296014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecedc4e5ddb643e0b0852dc72cfdcc8c)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:53.297964 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:53.298558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d1df9-f2e6-4bd7-b3c8-30cbff5afbe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:53.299211 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d1df9-f2e6-4bd7-b3c8-30cbff5afbe4 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:53.299490 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d1df9-f2e6-4bd7-b3c8-30cbff5afbe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:53.299761 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d1df9-f2e6-4bd7-b3c8-30cbff5afbe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:53.301617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-9a7d1df9-f2e6-4bd7-b3c8-30cbff5afbe4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:53.345159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecedc4e5ddb643e0b0852dc72cfdcc8c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.345814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecedc4e5ddb643e0b0852dc72cfdcc8c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.346419 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecedc4e5ddb643e0b0852dc72cfdcc8c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.354399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecedc4e5ddb643e0b0852dc72cfdcc8c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.361622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d1df9-f2e6-4bd7-b3c8-30cbff5afbe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:53.362195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecedc4e5ddb643e0b0852dc72cfdcc8c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.363073 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d1df9-f2e6-4bd7-b3c8-30cbff5afbe4 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:53.374685 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1329/2620] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:53 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:53.376372 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d6fae-0c39-4f59-b884-160f598d57ee None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:53.376668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d6fae-0c39-4f59-b884-160f598d57ee None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:53.376947 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d6fae-0c39-4f59-b884-160f598d57ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:53.394198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1292/2621] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:53 2022] DELETE /identity/v3/users/ecedc4e5ddb643e0b0852dc72cfdcc8c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:53.398375 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-5a7d6fae-0c39-4f59-b884-160f598d57ee None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:53.399717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5a7d6fae-0c39-4f59-b884-160f598d57ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:53.468383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1330/2622] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:53.480253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:53.482648 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:53.555444 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513e1ca-12b2-433d-9d6d-fe7c0b261d0e admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:53.559804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd444d9f-cd45-4f33-bf49-64ef7c74ceae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:53.561304 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd444d9f-cd45-4f33-bf49-64ef7c74ceae service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:53.562142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd444d9f-cd45-4f33-bf49-64ef7c74ceae service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:53.562611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd444d9f-cd45-4f33-bf49-64ef7c74ceae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:53.573380 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd444d9f-cd45-4f33-bf49-64ef7c74ceae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:53.574765 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd444d9f-cd45-4f33-bf49-64ef7c74ceae service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:53.590277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1293/2623] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:53.788505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d6fae-0c39-4f59-b884-160f598d57ee None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:53.790600 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5a7d6fae-0c39-4f59-b884-160f598d57ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:53.807252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d6fae-0c39-4f59-b884-160f598d57ee None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:53.814085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87015e-47c4-4620-8e0f-34238daa8536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:53.814986 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87015e-47c4-4620-8e0f-34238daa8536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:53.815265 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87015e-47c4-4620-8e0f-34238daa8536 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:53.815540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87015e-47c4-4620-8e0f-34238daa8536 admin admin] PATH_INFO: `/v3/projects/c6c66198e48543b7ae9e0dfa0ca320a9` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:53.816866 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87015e-47c4-4620-8e0f-34238daa8536 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6c66198e48543b7ae9e0dfa0ca320a9)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:53.819378 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87015e-47c4-4620-8e0f-34238daa8536 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:53.846431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa59eb3-4960-41fe-b365-88175b9f0b5a None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:53.846561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa59eb3-4960-41fe-b365-88175b9f0b5a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:53.846829 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa59eb3-4960-41fe-b365-88175b9f0b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:53.869188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-cfa59eb3-4960-41fe-b365-88175b9f0b5a None None] MFA Rules not processed for user `cb34feb512764ec5b9ca25c30c3d7159`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:53.870907 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-cfa59eb3-4960-41fe-b365-88175b9f0b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:53.896973 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-dc87015e-47c4-4620-8e0f-34238daa8536 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6c66198e48543b7ae9e0dfa0ca320a9', 'request_id': 'req-dc87015e-47c4-4620-8e0f-34238daa8536'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.902657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-dc87015e-47c4-4620-8e0f-34238daa8536 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6c66198e48543b7ae9e0dfa0ca320a9', 'request_id': 'req-dc87015e-47c4-4620-8e0f-34238daa8536'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.913892 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-dc87015e-47c4-4620-8e0f-34238daa8536 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6c66198e48543b7ae9e0dfa0ca320a9', 'request_id': 'req-dc87015e-47c4-4620-8e0f-34238daa8536'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.927504 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-dc87015e-47c4-4620-8e0f-34238daa8536 admin admin] The token cache is being invalidate because project c6c66198e48543b7ae9e0dfa0ca320a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:53.927836 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-dc87015e-47c4-4620-8e0f-34238daa8536 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.928472 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-dc87015e-47c4-4620-8e0f-34238daa8536 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:53.930041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1331/2624] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:53 2022] DELETE /identity/v3/projects/c6c66198e48543b7ae9e0dfa0ca320a9 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:53.936309 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb0ddf-11dc-48a3-b5bd-5e3b0763122b None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:53.936594 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb0ddf-11dc-48a3-b5bd-5e3b0763122b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:53.936873 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb0ddf-11dc-48a3-b5bd-5e3b0763122b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:53.938154 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1294/2625] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:33:53.965268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-00cb0ddf-11dc-48a3-b5bd-5e3b0763122b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:53.967250 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-00cb0ddf-11dc-48a3-b5bd-5e3b0763122b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:54.007334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa59eb3-4960-41fe-b365-88175b9f0b5a None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:54.009278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-cfa59eb3-4960-41fe-b365-88175b9f0b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:54.026598 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa59eb3-4960-41fe-b365-88175b9f0b5a None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:54.055176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1332/2626] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:54.063597 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a095354-ea63-421d-b0d7-419c158feaaf None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.063957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a095354-ea63-421d-b0d7-419c158feaaf None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.064339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a095354-ea63-421d-b0d7-419c158feaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.082651 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-3a095354-ea63-421d-b0d7-419c158feaaf None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:54.084037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3a095354-ea63-421d-b0d7-419c158feaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:54.093083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:54.094307 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.094590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.095126 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] PATH_INFO: `/v3/projects/2379c1ea639149f0b4a421a39faa18be` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.096035 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1333/2627] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:54.096157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=2379c1ea639149f0b4a421a39faa18be)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:54.097986 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:54.098142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a095354-ea63-421d-b0d7-419c158feaaf None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:54.112243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a095354-ea63-421d-b0d7-419c158feaaf None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:54.117346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a49c00-b165-4d1a-aec8-e8aa4db5e8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:54.118358 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a49c00-b165-4d1a-aec8-e8aa4db5e8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.118641 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a49c00-b165-4d1a-aec8-e8aa4db5e8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.118917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a49c00-b165-4d1a-aec8-e8aa4db5e8e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.119778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a49c00-b165-4d1a-aec8-e8aa4db5e8e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:33:54.120164 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a49c00-b165-4d1a-aec8-e8aa4db5e8e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:54.122143 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a49c00-b165-4d1a-aec8-e8aa4db5e8e7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:54.135719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1334/2628] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:33:54.140234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a49c00-b165-4d1a-aec8-e8aa4db5e8e7 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:54.148690 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2379c1ea639149f0b4a421a39faa18be', 'request_id': 'req-86644945-b247-4b8f-92a6-675f90a38eed'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.155594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2379c1ea639149f0b4a421a39faa18be', 'request_id': 'req-86644945-b247-4b8f-92a6-675f90a38eed'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.161337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2379c1ea639149f0b4a421a39faa18be', 'request_id': 'req-86644945-b247-4b8f-92a6-675f90a38eed'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.166301 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a49c00-b165-4d1a-aec8-e8aa4db5e8e7 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:54.176472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] The token cache is being invalidate because project 2379c1ea639149f0b4a421a39faa18be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:54.176999 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.177666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.179482 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1295/2629] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:54 2022] DELETE /identity/v3/projects/2379c1ea639149f0b4a421a39faa18be => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:54.185730 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:54.188032 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:54.247645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01780f4-d74d-4bef-a022-3356727b4aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:54.249238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01780f4-d74d-4bef-a022-3356727b4aec admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.249528 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01780f4-d74d-4bef-a022-3356727b4aec admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.249806 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01780f4-d74d-4bef-a022-3356727b4aec admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.250511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01780f4-d74d-4bef-a022-3356727b4aec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:54.252443 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01780f4-d74d-4bef-a022-3356727b4aec admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:54.276607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86644945-b247-4b8f-92a6-675f90a38eed admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:54.281963 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:54.282840 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.283120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.283677 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] PATH_INFO: `/v3/users/96beefe6b93540e7a5fff98d47af8311` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.283792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1335/2630] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:54 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:54.284683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96beefe6b93540e7a5fff98d47af8311)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:54.287439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:54.287569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01780f4-d74d-4bef-a022-3356727b4aec admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:54.289320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b01780f4-d74d-4bef-a022-3356727b4aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:54.340060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96beefe6b93540e7a5fff98d47af8311'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.340728 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96beefe6b93540e7a5fff98d47af8311'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.341459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96beefe6b93540e7a5fff98d47af8311'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.349482 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96beefe6b93540e7a5fff98d47af8311'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.356341 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96beefe6b93540e7a5fff98d47af8311'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.369831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01780f4-d74d-4bef-a022-3356727b4aec admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:54.375157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c724f6-dd81-41c6-96b3-58e77232a41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:54.375375 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1296/2631] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:33:54 2022] DELETE /identity/v3/users/96beefe6b93540e7a5fff98d47af8311 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:54.378583 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c724f6-dd81-41c6-96b3-58e77232a41d admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.378583 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c724f6-dd81-41c6-96b3-58e77232a41d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.378830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:54.379051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c724f6-dd81-41c6-96b3-58e77232a41d admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.379051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c724f6-dd81-41c6-96b3-58e77232a41d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:54.379259 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:54.381002 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c724f6-dd81-41c6-96b3-58e77232a41d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:54.403723 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1336/2632] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:54 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:33:54.407549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c724f6-dd81-41c6-96b3-58e77232a41d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:54.409247 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-69c724f6-dd81-41c6-96b3-58e77232a41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:54.424351 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c724f6-dd81-41c6-96b3-58e77232a41d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:54.439150 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8987a4-9f5c-4553-8592-75f5846b2768 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:54.460356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2015221-d9aa-48ef-ae38-6870d9be907e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:54.461021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2015221-d9aa-48ef-ae38-6870d9be907e service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.461310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2015221-d9aa-48ef-ae38-6870d9be907e service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.461590 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2015221-d9aa-48ef-ae38-6870d9be907e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.463445 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d2015221-d9aa-48ef-ae38-6870d9be907e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:54.474025 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2015221-d9aa-48ef-ae38-6870d9be907e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:54.475856 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2015221-d9aa-48ef-ae38-6870d9be907e service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:54.486804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1297/2633] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:54.488803 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb318453-db68-41b1-9042-e95b19db5319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:54.489480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb318453-db68-41b1-9042-e95b19db5319 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.489759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb318453-db68-41b1-9042-e95b19db5319 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.490031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb318453-db68-41b1-9042-e95b19db5319 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.490762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb318453-db68-41b1-9042-e95b19db5319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:54.492384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb318453-db68-41b1-9042-e95b19db5319 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:54.505831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1337/2634] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:33:54.506979 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2015221-d9aa-48ef-ae38-6870d9be907e service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:54.518205 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2015221-d9aa-48ef-ae38-6870d9be907e service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:54.522982 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ebf48-1d63-40d8-a8fa-538b0c7204ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:54.523642 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ebf48-1d63-40d8-a8fa-538b0c7204ca admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.523955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ebf48-1d63-40d8-a8fa-538b0c7204ca admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.524235 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ebf48-1d63-40d8-a8fa-538b0c7204ca admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.524951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ebf48-1d63-40d8-a8fa-538b0c7204ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:54.526525 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ebf48-1d63-40d8-a8fa-538b0c7204ca admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:54.539808 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1298/2635] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:54.541046 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb318453-db68-41b1-9042-e95b19db5319 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:54.551856 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb318453-db68-41b1-9042-e95b19db5319 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:54.556749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55404d8e-65f2-4d83-929d-48ad4fba5d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:54.557701 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55404d8e-65f2-4d83-929d-48ad4fba5d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.558155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55404d8e-65f2-4d83-929d-48ad4fba5d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.558328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55404d8e-65f2-4d83-929d-48ad4fba5d22 admin admin] PATH_INFO: `/v3/projects/20b03a05541e4ee38dc5f94a6e4bdc69/users/09dd259fed464b8c98749dd7e369e871/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.573592 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55404d8e-65f2-4d83-929d-48ad4fba5d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b03a05541e4ee38dc5f94a6e4bdc69, user_id=09dd259fed464b8c98749dd7e369e871, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:54.576714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55404d8e-65f2-4d83-929d-48ad4fba5d22 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:54.594586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1338/2636] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:54 2022] PUT /identity/v3/projects/20b03a05541e4ee38dc5f94a6e4bdc69/users/09dd259fed464b8c98749dd7e369e871/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:54.605350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09c702-c062-4041-9696-8e5cff93fe86 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.605350 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09c702-c062-4041-9696-8e5cff93fe86 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.605672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09c702-c062-4041-9696-8e5cff93fe86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.632015 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-7a09c702-c062-4041-9696-8e5cff93fe86 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:54.633783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-7a09c702-c062-4041-9696-8e5cff93fe86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:54.702460 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1299/2637] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:54.712071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55404d8e-65f2-4d83-929d-48ad4fba5d22 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:54.713882 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-55404d8e-65f2-4d83-929d-48ad4fba5d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:54.729162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09c702-c062-4041-9696-8e5cff93fe86 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:54.730596 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-7a09c702-c062-4041-9696-8e5cff93fe86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:54.744729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09c702-c062-4041-9696-8e5cff93fe86 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:54.770754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9f8e4-fec5-4b70-b142-dfe900348afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:54.773422 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9f8e4-fec5-4b70-b142-dfe900348afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.773965 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9f8e4-fec5-4b70-b142-dfe900348afd admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.774646 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9f8e4-fec5-4b70-b142-dfe900348afd admin admin] PATH_INFO: `/v3/projects/a7de06aa34ae489e9671f220b5ee13da` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.775996 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9f8e4-fec5-4b70-b142-dfe900348afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7de06aa34ae489e9671f220b5ee13da)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:54.777917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9f8e4-fec5-4b70-b142-dfe900348afd admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:54.832757 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55404d8e-65f2-4d83-929d-48ad4fba5d22 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:54.845439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-92d9f8e4-fec5-4b70-b142-dfe900348afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7de06aa34ae489e9671f220b5ee13da', 'request_id': 'req-92d9f8e4-fec5-4b70-b142-dfe900348afd'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.853098 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-92d9f8e4-fec5-4b70-b142-dfe900348afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7de06aa34ae489e9671f220b5ee13da', 'request_id': 'req-92d9f8e4-fec5-4b70-b142-dfe900348afd'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.862553 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-92d9f8e4-fec5-4b70-b142-dfe900348afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7de06aa34ae489e9671f220b5ee13da', 'request_id': 'req-92d9f8e4-fec5-4b70-b142-dfe900348afd'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.888062 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-92d9f8e4-fec5-4b70-b142-dfe900348afd admin admin] The token cache is being invalidate because project a7de06aa34ae489e9671f220b5ee13da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:33:54.888501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-92d9f8e4-fec5-4b70-b142-dfe900348afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.889212 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-92d9f8e4-fec5-4b70-b142-dfe900348afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:33:54.892232 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1300/2638] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:33:54 2022] DELETE /identity/v3/projects/a7de06aa34ae489e9671f220b5ee13da => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:54.902259 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203abe3-7ff5-487c-a5f0-4fd0fd40d21f None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.903581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203abe3-7ff5-487c-a5f0-4fd0fd40d21f None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.904512 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203abe3-7ff5-487c-a5f0-4fd0fd40d21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.907102 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da4cc1-cc35-44be-b9cf-58963af3961f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:54.909303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da4cc1-cc35-44be-b9cf-58963af3961f service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:54.910326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da4cc1-cc35-44be-b9cf-58963af3961f service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:54.910835 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da4cc1-cc35-44be-b9cf-58963af3961f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:54.913989 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-66da4cc1-cc35-44be-b9cf-58963af3961f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:54.932228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-0203abe3-7ff5-487c-a5f0-4fd0fd40d21f None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:54.933495 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0203abe3-7ff5-487c-a5f0-4fd0fd40d21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:54.992924 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da4cc1-cc35-44be-b9cf-58963af3961f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:54.994370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da4cc1-cc35-44be-b9cf-58963af3961f service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:55.006788 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1339/2639] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:33:55.011439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1301/2640] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:55.020257 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538890e-d0e0-4736-ae4d-3f2d37affa9b None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:55.020559 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538890e-d0e0-4736-ae4d-3f2d37affa9b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:55.020953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538890e-d0e0-4736-ae4d-3f2d37affa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:55.058303 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-b538890e-d0e0-4736-ae4d-3f2d37affa9b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:55.062264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b538890e-d0e0-4736-ae4d-3f2d37affa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:55.077120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1340/2641] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:55.080534 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203abe3-7ff5-487c-a5f0-4fd0fd40d21f None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:55.093435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203abe3-7ff5-487c-a5f0-4fd0fd40d21f None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:55.101086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04764f2-55d5-4ca2-9d7e-368d1eecc2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:55.101773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04764f2-55d5-4ca2-9d7e-368d1eecc2be admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:55.102439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04764f2-55d5-4ca2-9d7e-368d1eecc2be admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:55.102439 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04764f2-55d5-4ca2-9d7e-368d1eecc2be admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:55.103687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04764f2-55d5-4ca2-9d7e-368d1eecc2be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:33:55.104057 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04764f2-55d5-4ca2-9d7e-368d1eecc2be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:55.105811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04764f2-55d5-4ca2-9d7e-368d1eecc2be admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:55.127184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1302/2642] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:33:55.128104 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538890e-d0e0-4736-ae4d-3f2d37affa9b None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:55.139865 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538890e-d0e0-4736-ae4d-3f2d37affa9b None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:55.146237 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e55d1b-5987-4c71-b9c7-f062ac75b8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:55.147086 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e55d1b-5987-4c71-b9c7-f062ac75b8ae admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:55.147493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e55d1b-5987-4c71-b9c7-f062ac75b8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:55.147911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e55d1b-5987-4c71-b9c7-f062ac75b8ae admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:55.148762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e55d1b-5987-4c71-b9c7-f062ac75b8ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:55.150958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e55d1b-5987-4c71-b9c7-f062ac75b8ae admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:55.233762 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1341/2643] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:33:55 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:55.237989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04764f2-55d5-4ca2-9d7e-368d1eecc2be admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:55.260936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04764f2-55d5-4ca2-9d7e-368d1eecc2be admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:55.365655 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01d45d-8a26-4539-848a-9500784caab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:55.366660 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d45d-8a26-4539-848a-9500784caab8 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:55.367749 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d45d-8a26-4539-848a-9500784caab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:55.368596 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d45d-8a26-4539-848a-9500784caab8 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:55.370836 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01d45d-8a26-4539-848a-9500784caab8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:55.374129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01d45d-8a26-4539-848a-9500784caab8 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:55.403490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1303/2644] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:33:55 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:33:55.406659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e55d1b-5987-4c71-b9c7-f062ac75b8ae admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:55.417403 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e55d1b-5987-4c71-b9c7-f062ac75b8ae admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:55.422505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46421a9-233b-4895-a526-9f065b470f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:55.423386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46421a9-233b-4895-a526-9f065b470f2b admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:55.423734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46421a9-233b-4895-a526-9f065b470f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:55.424144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46421a9-233b-4895-a526-9f065b470f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:55.425508 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46421a9-233b-4895-a526-9f065b470f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:55.426776 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46421a9-233b-4895-a526-9f065b470f2b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:55.439637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1342/2645] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:55.443768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01d45d-8a26-4539-848a-9500784caab8 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:55.456607 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01d45d-8a26-4539-848a-9500784caab8 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:55.461512 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cbe97-42d3-4499-b9e9-29039cfe396e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:55.462186 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cbe97-42d3-4499-b9e9-29039cfe396e admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:55.462470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cbe97-42d3-4499-b9e9-29039cfe396e admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:55.462768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cbe97-42d3-4499-b9e9-29039cfe396e admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:55.463468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cbe97-42d3-4499-b9e9-29039cfe396e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:55.465334 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cbe97-42d3-4499-b9e9-29039cfe396e admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:55.476560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1304/2646] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:33:55.480630 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46421a9-233b-4895-a526-9f065b470f2b admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:55.491803 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46421a9-233b-4895-a526-9f065b470f2b admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:55.500435 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1ae54-fc62-4fc6-ba75-c9c62a4e7d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:55.501350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1ae54-fc62-4fc6-ba75-c9c62a4e7d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:55.501637 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1ae54-fc62-4fc6-ba75-c9c62a4e7d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:55.501914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1ae54-fc62-4fc6-ba75-c9c62a4e7d28 admin admin] PATH_INFO: `/v3/projects/b765dad498034654800d9a668398a296/users/a7be5a11e4d2458897ece6b9b04fded9/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:55.519589 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1ae54-fc62-4fc6-ba75-c9c62a4e7d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b765dad498034654800d9a668398a296, user_id=a7be5a11e4d2458897ece6b9b04fded9, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:55.522291 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1ae54-fc62-4fc6-ba75-c9c62a4e7d28 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:55.543566 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1343/2647] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:33:55 2022] PUT /identity/v3/projects/b765dad498034654800d9a668398a296/users/a7be5a11e4d2458897ece6b9b04fded9/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:33:55.554627 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c5e70-aa94-458e-b86c-c695d7b78ecd None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:55.555372 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c5e70-aa94-458e-b86c-c695d7b78ecd None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:55.555838 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c5e70-aa94-458e-b86c-c695d7b78ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:55.594636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-451c5e70-aa94-458e-b86c-c695d7b78ecd None None] MFA Rules not processed for user `a7be5a11e4d2458897ece6b9b04fded9`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:55.596172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-451c5e70-aa94-458e-b86c-c695d7b78ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:55.685760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1305/2648] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:55.697157 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1ae54-fc62-4fc6-ba75-c9c62a4e7d28 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:55.699047 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-0ad1ae54-fc62-4fc6-ba75-c9c62a4e7d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:55.764875 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1ae54-fc62-4fc6-ba75-c9c62a4e7d28 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:55.769090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befd07a-1047-4306-99e6-35289b529bdb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '295bb46627e043989e6dba2805607c99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:55.770047 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befd07a-1047-4306-99e6-35289b529bdb service glance] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:55.770221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befd07a-1047-4306-99e6-35289b529bdb service glance] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:55.770607 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befd07a-1047-4306-99e6-35289b529bdb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:55.780855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befd07a-1047-4306-99e6-35289b529bdb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:55.782405 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befd07a-1047-4306-99e6-35289b529bdb service glance] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:55.792884 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1344/2649] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:33:55.910370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe336efb-df82-48b4-b94a-f41ad2e555f8 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:55.910732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe336efb-df82-48b4-b94a-f41ad2e555f8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:55.911459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe336efb-df82-48b4-b94a-f41ad2e555f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:55.949198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-fe336efb-df82-48b4-b94a-f41ad2e555f8 None None] MFA Rules not processed for user `09dd259fed464b8c98749dd7e369e871`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:55.951149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-fe336efb-df82-48b4-b94a-f41ad2e555f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:56.027199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1306/2650] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:33:56.038385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befd07a-1047-4306-99e6-35289b529bdb service glance] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:33:56.039914 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5befd07a-1047-4306-99e6-35289b529bdb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:56.118475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befd07a-1047-4306-99e6-35289b529bdb service glance] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:33:56.124205 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201c4a1-743f-4509-9624-6b46cd32bfe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:33:56.124941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201c4a1-743f-4509-9624-6b46cd32bfe2 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:56.125631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201c4a1-743f-4509-9624-6b46cd32bfe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:56.125631 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201c4a1-743f-4509-9624-6b46cd32bfe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:56.135493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201c4a1-743f-4509-9624-6b46cd32bfe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:33:56.137059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201c4a1-743f-4509-9624-6b46cd32bfe2 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:33:56.146911 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1345/2651] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:33:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:33:57.789926 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cf25a-c2db-494a-8609-ff74b2b7486f None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:33:57.790400 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cf25a-c2db-494a-8609-ff74b2b7486f None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:33:57.790484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cf25a-c2db-494a-8609-ff74b2b7486f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:33:57.808498 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-942cf25a-c2db-494a-8609-ff74b2b7486f None None] MFA Rules not processed for user `09dd259fed464b8c98749dd7e369e871`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:33:57.809700 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-942cf25a-c2db-494a-8609-ff74b2b7486f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:33:57.817369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1307/2652] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:33:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:02.218121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201c4a1-743f-4509-9624-6b46cd32bfe2 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:02.219537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e201c4a1-743f-4509-9624-6b46cd32bfe2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:02.283183 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201c4a1-743f-4509-9624-6b46cd32bfe2 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:02.287628 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:02.288624 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:02.288903 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:02.289177 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] PATH_INFO: `/v3/users/674f29fe1441416f9a22e96fa82d35e3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:02.290373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] RBAC: Authorizing `identity:delete_user(user_id=674f29fe1441416f9a22e96fa82d35e3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:02.292340 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:02.337422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '674f29fe1441416f9a22e96fa82d35e3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:02.338061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '674f29fe1441416f9a22e96fa82d35e3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:02.345609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '674f29fe1441416f9a22e96fa82d35e3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:02.352576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '674f29fe1441416f9a22e96fa82d35e3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:02.353241 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '674f29fe1441416f9a22e96fa82d35e3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:02.369452 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1346/2653] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:34:02 2022] DELETE /identity/v3/users/674f29fe1441416f9a22e96fa82d35e3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:02.374033 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0e59b-34fc-4b6a-8647-e7615d875737 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:02.374509 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0e59b-34fc-4b6a-8647-e7615d875737 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:02.374678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0e59b-34fc-4b6a-8647-e7615d875737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:02.393348 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-50c0e59b-34fc-4b6a-8647-e7615d875737 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:02.395027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-50c0e59b-34fc-4b6a-8647-e7615d875737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:02.456412 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1308/2654] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:02.467874 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:02.469503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:02.525124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198652f1-e44a-4434-8775-c41e3ada0618 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:02.528781 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc094a8-4eba-46c3-9d28-fe6287b40ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:02.529432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc094a8-4eba-46c3-9d28-fe6287b40ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:02.529764 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc094a8-4eba-46c3-9d28-fe6287b40ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:02.530577 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc094a8-4eba-46c3-9d28-fe6287b40ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:02.541725 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc094a8-4eba-46c3-9d28-fe6287b40ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:02.543205 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc094a8-4eba-46c3-9d28-fe6287b40ea6 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:02.553004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1347/2655] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:02.866351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0e59b-34fc-4b6a-8647-e7615d875737 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:02.868142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-50c0e59b-34fc-4b6a-8647-e7615d875737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:02.887242 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0e59b-34fc-4b6a-8647-e7615d875737 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:02.892467 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb21611-5a20-4d20-8184-28ebd52d150c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:02.893446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb21611-5a20-4d20-8184-28ebd52d150c admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:02.893830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb21611-5a20-4d20-8184-28ebd52d150c admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:02.894059 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb21611-5a20-4d20-8184-28ebd52d150c admin admin] PATH_INFO: `/v3/projects/5eaf003451d34d28b0795a072e96a44c` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:02.895261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb21611-5a20-4d20-8184-28ebd52d150c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eaf003451d34d28b0795a072e96a44c)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:02.897086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb21611-5a20-4d20-8184-28ebd52d150c admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:02.944451 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2eb21611-5a20-4d20-8184-28ebd52d150c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eaf003451d34d28b0795a072e96a44c', 'request_id': 'req-2eb21611-5a20-4d20-8184-28ebd52d150c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:02.950577 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2eb21611-5a20-4d20-8184-28ebd52d150c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eaf003451d34d28b0795a072e96a44c', 'request_id': 'req-2eb21611-5a20-4d20-8184-28ebd52d150c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:02.957220 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2eb21611-5a20-4d20-8184-28ebd52d150c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eaf003451d34d28b0795a072e96a44c', 'request_id': 'req-2eb21611-5a20-4d20-8184-28ebd52d150c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:02.979657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2eb21611-5a20-4d20-8184-28ebd52d150c admin admin] The token cache is being invalidate because project 5eaf003451d34d28b0795a072e96a44c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:34:02.979974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2eb21611-5a20-4d20-8184-28ebd52d150c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:02.980631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2eb21611-5a20-4d20-8184-28ebd52d150c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:02.982283 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1309/2656] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:34:02 2022] DELETE /identity/v3/projects/5eaf003451d34d28b0795a072e96a44c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:02.988172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f71b05-139e-4001-8bf5-0e72c404e8f6 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:02.988466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f71b05-139e-4001-8bf5-0e72c404e8f6 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:02.988748 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f71b05-139e-4001-8bf5-0e72c404e8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.006277 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-04f71b05-139e-4001-8bf5-0e72c404e8f6 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:03.007934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-04f71b05-139e-4001-8bf5-0e72c404e8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:03.066461 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1348/2657] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:03.075670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5840d-e91a-4472-b7be-cdf763c14c8a None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.075995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5840d-e91a-4472-b7be-cdf763c14c8a None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.076276 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5840d-e91a-4472-b7be-cdf763c14c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.095482 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-c8a5840d-e91a-4472-b7be-cdf763c14c8a None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:03.096732 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c8a5840d-e91a-4472-b7be-cdf763c14c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:03.104566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1310/2658] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:03.109377 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f71b05-139e-4001-8bf5-0e72c404e8f6 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:03.119076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f71b05-139e-4001-8bf5-0e72c404e8f6 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:03.126249 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5751d-60e5-48d9-96c9-296e4a496f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:03.126961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5751d-60e5-48d9-96c9-296e4a496f89 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.127262 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5751d-60e5-48d9-96c9-296e4a496f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.127521 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5751d-60e5-48d9-96c9-296e4a496f89 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.128299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5751d-60e5-48d9-96c9-296e4a496f89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:34:03.128604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5751d-60e5-48d9-96c9-296e4a496f89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:03.130234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5751d-60e5-48d9-96c9-296e4a496f89 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:03.142954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1349/2659] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:34:03.147364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5840d-e91a-4472-b7be-cdf763c14c8a None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:03.157323 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5840d-e91a-4472-b7be-cdf763c14c8a None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:03.161967 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f72594-78da-4725-afc4-04b3d02dd565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:03.162611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f72594-78da-4725-afc4-04b3d02dd565 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.162895 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f72594-78da-4725-afc4-04b3d02dd565 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.163173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f72594-78da-4725-afc4-04b3d02dd565 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.163866 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f72594-78da-4725-afc4-04b3d02dd565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:03.165760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f72594-78da-4725-afc4-04b3d02dd565 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:03.195231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1311/2660] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:34:03 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:03.199505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5751d-60e5-48d9-96c9-296e4a496f89 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:03.209618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5751d-60e5-48d9-96c9-296e4a496f89 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:03.268575 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c8ac3-bbc8-4e45-8832-cd8ad074a951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:03.269221 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c8ac3-bbc8-4e45-8832-cd8ad074a951 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.269474 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c8ac3-bbc8-4e45-8832-cd8ad074a951 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.269747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c8ac3-bbc8-4e45-8832-cd8ad074a951 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.270455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c8ac3-bbc8-4e45-8832-cd8ad074a951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:03.272387 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c8ac3-bbc8-4e45-8832-cd8ad074a951 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:03.291812 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1350/2661] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:34:03 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:03.296461 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f72594-78da-4725-afc4-04b3d02dd565 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:03.306763 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f72594-78da-4725-afc4-04b3d02dd565 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:03.312014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c12d3-ca95-4301-8647-96cb2b4dc9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:03.312671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c12d3-ca95-4301-8647-96cb2b4dc9c2 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.312951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c12d3-ca95-4301-8647-96cb2b4dc9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.313224 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c12d3-ca95-4301-8647-96cb2b4dc9c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.313998 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c12d3-ca95-4301-8647-96cb2b4dc9c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:03.315557 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c12d3-ca95-4301-8647-96cb2b4dc9c2 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:03.325552 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1312/2662] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:03.329415 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c8ac3-bbc8-4e45-8832-cd8ad074a951 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:03.338978 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c8ac3-bbc8-4e45-8832-cd8ad074a951 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:03.343742 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ab8a2-5fb2-470b-8649-e9ae388729f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:03.344449 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ab8a2-5fb2-470b-8649-e9ae388729f7 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.344732 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ab8a2-5fb2-470b-8649-e9ae388729f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.345015 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ab8a2-5fb2-470b-8649-e9ae388729f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.345735 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ab8a2-5fb2-470b-8649-e9ae388729f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:03.347311 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ab8a2-5fb2-470b-8649-e9ae388729f7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:03.357335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1351/2663] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:03.362241 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c12d3-ca95-4301-8647-96cb2b4dc9c2 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:03.372434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c12d3-ca95-4301-8647-96cb2b4dc9c2 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:03.379151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cb526-311a-43dd-b487-9fe6c1347cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:03.380246 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cb526-311a-43dd-b487-9fe6c1347cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.380555 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cb526-311a-43dd-b487-9fe6c1347cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.380819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635cb526-311a-43dd-b487-9fe6c1347cb7 admin admin] PATH_INFO: `/v3/projects/9c0a7df74d0f4d31b8f07a8c81384d0d/users/303945780cac4da882ec99992f33fdd9/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.396069 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635cb526-311a-43dd-b487-9fe6c1347cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c0a7df74d0f4d31b8f07a8c81384d0d, user_id=303945780cac4da882ec99992f33fdd9, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:03.398701 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635cb526-311a-43dd-b487-9fe6c1347cb7 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:03.412443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1313/2664] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:03 2022] PUT /identity/v3/projects/9c0a7df74d0f4d31b8f07a8c81384d0d/users/303945780cac4da882ec99992f33fdd9/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:03.423635 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e21fb4-d407-46d9-b55b-687449ec6f66 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.423969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e21fb4-d407-46d9-b55b-687449ec6f66 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.424256 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e21fb4-d407-46d9-b55b-687449ec6f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.451016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-92e21fb4-d407-46d9-b55b-687449ec6f66 None None] MFA Rules not processed for user `303945780cac4da882ec99992f33fdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:03.452723 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-92e21fb4-d407-46d9-b55b-687449ec6f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:03.521523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1352/2665] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:03.525882 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cb526-311a-43dd-b487-9fe6c1347cb7 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:03.587430 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635cb526-311a-43dd-b487-9fe6c1347cb7 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:03.592752 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4995ef1-b4c6-4e06-bef6-ef227742483d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:03.593585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4995ef1-b4c6-4e06-bef6-ef227742483d admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.593866 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4995ef1-b4c6-4e06-bef6-ef227742483d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.594137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4995ef1-b4c6-4e06-bef6-ef227742483d admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.594827 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4995ef1-b4c6-4e06-bef6-ef227742483d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:03.596745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4995ef1-b4c6-4e06-bef6-ef227742483d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:03.622736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1314/2666] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:34:03 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:34:03.630044 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e21fb4-d407-46d9-b55b-687449ec6f66 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:03.642521 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e21fb4-d407-46d9-b55b-687449ec6f66 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:03.713721 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2fa00-4282-4a14-b529-623b1e158aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:03.714745 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2fa00-4282-4a14-b529-623b1e158aad admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.715107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2fa00-4282-4a14-b529-623b1e158aad admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.715451 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2fa00-4282-4a14-b529-623b1e158aad admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.716252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2fa00-4282-4a14-b529-623b1e158aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:03.719020 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2fa00-4282-4a14-b529-623b1e158aad admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:03.758372 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1353/2667] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:34:03 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:34:03.764177 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4995ef1-b4c6-4e06-bef6-ef227742483d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:03.777175 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4995ef1-b4c6-4e06-bef6-ef227742483d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:03.784078 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01df89-9561-4398-b1ad-d802fd429462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:03.784760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01df89-9561-4398-b1ad-d802fd429462 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.785214 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01df89-9561-4398-b1ad-d802fd429462 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.785498 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01df89-9561-4398-b1ad-d802fd429462 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.786235 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01df89-9561-4398-b1ad-d802fd429462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:03.787813 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01df89-9561-4398-b1ad-d802fd429462 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:03.819953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1315/2668] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:34:03.821850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2fa00-4282-4a14-b529-623b1e158aad admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:03.833160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2fa00-4282-4a14-b529-623b1e158aad admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:03.838393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14353f-bb8e-4796-a478-70627b1cdbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:03.839749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14353f-bb8e-4796-a478-70627b1cdbb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.840030 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14353f-bb8e-4796-a478-70627b1cdbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.840144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14353f-bb8e-4796-a478-70627b1cdbb7 admin admin] PATH_INFO: `/v3/projects/627668c733174a2494bb0142dc06fd2c/users/368d760266d74f0c93f209708631d44c/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.856793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14353f-bb8e-4796-a478-70627b1cdbb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=627668c733174a2494bb0142dc06fd2c, user_id=368d760266d74f0c93f209708631d44c, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:03.859585 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14353f-bb8e-4796-a478-70627b1cdbb7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:03.874466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1354/2669] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:03 2022] PUT /identity/v3/projects/627668c733174a2494bb0142dc06fd2c/users/368d760266d74f0c93f209708631d44c/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:03.878377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01df89-9561-4398-b1ad-d802fd429462 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:03.888587 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01df89-9561-4398-b1ad-d802fd429462 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:03.948847 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e4722-7a0f-43ce-98ab-1a718647aa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:03.949955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e4722-7a0f-43ce-98ab-1a718647aa24 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.950314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e4722-7a0f-43ce-98ab-1a718647aa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.952359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e4722-7a0f-43ce-98ab-1a718647aa24 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.953239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e4722-7a0f-43ce-98ab-1a718647aa24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:03.954835 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e4722-7a0f-43ce-98ab-1a718647aa24 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:03.970486 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1316/2670] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:03.971861 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14353f-bb8e-4796-a478-70627b1cdbb7 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:03.982519 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14353f-bb8e-4796-a478-70627b1cdbb7 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:03.988749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3636ec0-a364-4742-acdc-4159bf37120b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:03.989734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3636ec0-a364-4742-acdc-4159bf37120b admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:03.990091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3636ec0-a364-4742-acdc-4159bf37120b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:03.990448 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3636ec0-a364-4742-acdc-4159bf37120b admin admin] PATH_INFO: `/v3/projects/627668c733174a2494bb0142dc06fd2c/users/368d760266d74f0c93f209708631d44c/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:03.993085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3636ec0-a364-4742-acdc-4159bf37120b admin admin] RBAC: Authorizing `identity:create_grant(project_id=627668c733174a2494bb0142dc06fd2c, user_id=368d760266d74f0c93f209708631d44c, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:03.995741 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3636ec0-a364-4742-acdc-4159bf37120b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:04.011948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1355/2671] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:03 2022] PUT /identity/v3/projects/627668c733174a2494bb0142dc06fd2c/users/368d760266d74f0c93f209708631d44c/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:04.018329 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7912ce-eb5e-4a6b-adae-88cd1cb469db None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:04.018619 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7912ce-eb5e-4a6b-adae-88cd1cb469db None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:04.018908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7912ce-eb5e-4a6b-adae-88cd1cb469db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:04.055689 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-6f7912ce-eb5e-4a6b-adae-88cd1cb469db None None] MFA Rules not processed for user `368d760266d74f0c93f209708631d44c`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:04.056981 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-6f7912ce-eb5e-4a6b-adae-88cd1cb469db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:04.128123 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1317/2672] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:04.141410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3636ec0-a364-4742-acdc-4159bf37120b admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:04.143114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c3636ec0-a364-4742-acdc-4159bf37120b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:04.205813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3636ec0-a364-4742-acdc-4159bf37120b admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:04.209896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3f5ad-c1aa-407c-91d0-836d04096e56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:04.210540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3f5ad-c1aa-407c-91d0-836d04096e56 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:04.210798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3f5ad-c1aa-407c-91d0-836d04096e56 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:04.211120 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3f5ad-c1aa-407c-91d0-836d04096e56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:04.222381 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3f5ad-c1aa-407c-91d0-836d04096e56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:04.223871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3f5ad-c1aa-407c-91d0-836d04096e56 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:04.236562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1356/2673] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:04.268651 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7912ce-eb5e-4a6b-adae-88cd1cb469db None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:04.334344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7912ce-eb5e-4a6b-adae-88cd1cb469db None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:04.338915 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:04.340076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:04.340359 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:04.340636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] PATH_INFO: `/v3/users/303945780cac4da882ec99992f33fdd9` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:04.341830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=303945780cac4da882ec99992f33fdd9)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:04.343784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:04.388959 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303945780cac4da882ec99992f33fdd9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:04.390594 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303945780cac4da882ec99992f33fdd9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:04.390773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303945780cac4da882ec99992f33fdd9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:04.397656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303945780cac4da882ec99992f33fdd9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:04.404428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303945780cac4da882ec99992f33fdd9'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:04.421023 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1318/2674] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:34:04 2022] DELETE /identity/v3/users/303945780cac4da882ec99992f33fdd9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:04.425519 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a97a41-a465-4375-b2a1-3af47d2359bc None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:04.425979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a97a41-a465-4375-b2a1-3af47d2359bc None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:04.426077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a97a41-a465-4375-b2a1-3af47d2359bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:04.448418 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-f4a97a41-a465-4375-b2a1-3af47d2359bc None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:04.449795 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f4a97a41-a465-4375-b2a1-3af47d2359bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:04.513349 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1357/2675] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:04.523533 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:04.525192 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:04.582789 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453f7bd-241c-4c71-9ad1-5dbd42de4674 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:04.586471 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2e92b-ea8d-4107-8f2c-ff20300137b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:04.587111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2e92b-ea8d-4107-8f2c-ff20300137b7 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:04.587407 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2e92b-ea8d-4107-8f2c-ff20300137b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:04.587671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2e92b-ea8d-4107-8f2c-ff20300137b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:04.597163 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2e92b-ea8d-4107-8f2c-ff20300137b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:04.598582 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2e92b-ea8d-4107-8f2c-ff20300137b7 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:04.608354 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1319/2676] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:04.882475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a97a41-a465-4375-b2a1-3af47d2359bc None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:04.884514 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f4a97a41-a465-4375-b2a1-3af47d2359bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:04.906814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a97a41-a465-4375-b2a1-3af47d2359bc None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:04.916413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:04.917861 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:04.918386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:04.918956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] PATH_INFO: `/v3/projects/9c0a7df74d0f4d31b8f07a8c81384d0d` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:04.921253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c0a7df74d0f4d31b8f07a8c81384d0d)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:04.923808 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:04.993654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c0a7df74d0f4d31b8f07a8c81384d0d', 'request_id': 'req-22acffb7-4056-4641-9b39-10bee0ae092f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.010095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c0a7df74d0f4d31b8f07a8c81384d0d', 'request_id': 'req-22acffb7-4056-4641-9b39-10bee0ae092f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.018839 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c0a7df74d0f4d31b8f07a8c81384d0d', 'request_id': 'req-22acffb7-4056-4641-9b39-10bee0ae092f'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.031075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] The token cache is being invalidate because project 9c0a7df74d0f4d31b8f07a8c81384d0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:34:05.031613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.032433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.034351 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1358/2677] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:34:04 2022] DELETE /identity/v3/projects/9c0a7df74d0f4d31b8f07a8c81384d0d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:34:05.038458 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2e92b-ea8d-4107-8f2c-ff20300137b7 service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:05.062366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2e92b-ea8d-4107-8f2c-ff20300137b7 service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:05.141599 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23d44b-a468-4f90-978c-21243440492b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:05.142502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23d44b-a468-4f90-978c-21243440492b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:05.142957 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23d44b-a468-4f90-978c-21243440492b admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:05.143415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23d44b-a468-4f90-978c-21243440492b admin admin] PATH_INFO: `/v3/users/368d760266d74f0c93f209708631d44c` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:05.144796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23d44b-a468-4f90-978c-21243440492b admin admin] RBAC: Authorizing `identity:delete_user(user_id=368d760266d74f0c93f209708631d44c)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:05.146735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23d44b-a468-4f90-978c-21243440492b admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:05.200387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0c23d44b-a468-4f90-978c-21243440492b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '368d760266d74f0c93f209708631d44c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.201029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0c23d44b-a468-4f90-978c-21243440492b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '368d760266d74f0c93f209708631d44c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.201699 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0c23d44b-a468-4f90-978c-21243440492b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '368d760266d74f0c93f209708631d44c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.209403 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0c23d44b-a468-4f90-978c-21243440492b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '368d760266d74f0c93f209708631d44c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.217757 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0c23d44b-a468-4f90-978c-21243440492b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '368d760266d74f0c93f209708631d44c'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.234441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1320/2678] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:34:05 2022] DELETE /identity/v3/users/368d760266d74f0c93f209708631d44c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:05.517065 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:05.518612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:05.532370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acffb7-4056-4641-9b39-10bee0ae092f admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:05.594955 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13637112-fa01-42f1-b15a-492c082b753b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:05.595821 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13637112-fa01-42f1-b15a-492c082b753b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:05.596124 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13637112-fa01-42f1-b15a-492c082b753b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:05.596392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13637112-fa01-42f1-b15a-492c082b753b admin admin] PATH_INFO: `/v3/projects/627668c733174a2494bb0142dc06fd2c` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:05.597562 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13637112-fa01-42f1-b15a-492c082b753b admin admin] RBAC: Authorizing `identity:delete_project(project_id=627668c733174a2494bb0142dc06fd2c)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:05.599446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13637112-fa01-42f1-b15a-492c082b753b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:05.648310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-13637112-fa01-42f1-b15a-492c082b753b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '627668c733174a2494bb0142dc06fd2c', 'request_id': 'req-13637112-fa01-42f1-b15a-492c082b753b'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.654253 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-13637112-fa01-42f1-b15a-492c082b753b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '627668c733174a2494bb0142dc06fd2c', 'request_id': 'req-13637112-fa01-42f1-b15a-492c082b753b'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.663052 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-13637112-fa01-42f1-b15a-492c082b753b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '627668c733174a2494bb0142dc06fd2c', 'request_id': 'req-13637112-fa01-42f1-b15a-492c082b753b'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.675946 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-13637112-fa01-42f1-b15a-492c082b753b admin admin] The token cache is being invalidate because project 627668c733174a2494bb0142dc06fd2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:34:05.676411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-13637112-fa01-42f1-b15a-492c082b753b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.676918 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-13637112-fa01-42f1-b15a-492c082b753b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:05.678847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1359/2679] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:34:05 2022] DELETE /identity/v3/projects/627668c733174a2494bb0142dc06fd2c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:05.684274 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f9ced-2b3a-417f-a165-13df0f57d747 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:05.684801 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f9ced-2b3a-417f-a165-13df0f57d747 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:05.693443 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f9ced-2b3a-417f-a165-13df0f57d747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:05.708168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-dd3f9ced-2b3a-417f-a165-13df0f57d747 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:05.709379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-dd3f9ced-2b3a-417f-a165-13df0f57d747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:05.771387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1321/2680] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:05.779072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1f910-28ca-4325-872f-39e2c77fd526 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:05.779362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1f910-28ca-4325-872f-39e2c77fd526 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:05.780203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1f910-28ca-4325-872f-39e2c77fd526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:05.800245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-8ab1f910-28ca-4325-872f-39e2c77fd526 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:05.801444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8ab1f910-28ca-4325-872f-39e2c77fd526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:05.809230 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1360/2681] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:05.813620 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f9ced-2b3a-417f-a165-13df0f57d747 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:05.823446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f9ced-2b3a-417f-a165-13df0f57d747 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:05.828172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395a425-c869-41b2-92d3-9a7a2e2406e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:05.828902 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395a425-c869-41b2-92d3-9a7a2e2406e0 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:05.829260 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395a425-c869-41b2-92d3-9a7a2e2406e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:05.829510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395a425-c869-41b2-92d3-9a7a2e2406e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:05.830190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395a425-c869-41b2-92d3-9a7a2e2406e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:34:05.830496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395a425-c869-41b2-92d3-9a7a2e2406e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:05.832117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395a425-c869-41b2-92d3-9a7a2e2406e0 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:05.847799 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1322/2682] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:34:05.852155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1f910-28ca-4325-872f-39e2c77fd526 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:05.861843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1f910-28ca-4325-872f-39e2c77fd526 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:05.866671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5651e8-b582-4c77-85f9-692e2802498b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:05.867408 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5651e8-b582-4c77-85f9-692e2802498b admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:05.867761 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5651e8-b582-4c77-85f9-692e2802498b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:05.868074 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5651e8-b582-4c77-85f9-692e2802498b admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:05.868769 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5651e8-b582-4c77-85f9-692e2802498b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:05.870745 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5651e8-b582-4c77-85f9-692e2802498b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:05.895784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1361/2683] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:34:05 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:05.899877 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395a425-c869-41b2-92d3-9a7a2e2406e0 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:05.911469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395a425-c869-41b2-92d3-9a7a2e2406e0 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:05.976437 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ea97b-bdf0-4502-8e9b-8bc6db795011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:05.977107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ea97b-bdf0-4502-8e9b-8bc6db795011 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:05.977557 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ea97b-bdf0-4502-8e9b-8bc6db795011 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:05.977980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ea97b-bdf0-4502-8e9b-8bc6db795011 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:05.978864 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ea97b-bdf0-4502-8e9b-8bc6db795011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:05.980738 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ea97b-bdf0-4502-8e9b-8bc6db795011 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:06.006086 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1323/2684] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:34:05 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:34:06.007234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5651e8-b582-4c77-85f9-692e2802498b admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:06.018475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5651e8-b582-4c77-85f9-692e2802498b admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:06.025804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc310d-6892-4f8a-a997-e4374055671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:06.025804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc310d-6892-4f8a-a997-e4374055671e admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:06.025804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc310d-6892-4f8a-a997-e4374055671e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:06.025804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc310d-6892-4f8a-a997-e4374055671e admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:06.026225 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc310d-6892-4f8a-a997-e4374055671e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:06.027517 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc310d-6892-4f8a-a997-e4374055671e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:06.039327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1362/2685] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:06.043442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ea97b-bdf0-4502-8e9b-8bc6db795011 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:06.055011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ea97b-bdf0-4502-8e9b-8bc6db795011 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:06.059852 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c171ca-9dc5-4f79-85f3-89564fdf5727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:06.060800 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c171ca-9dc5-4f79-85f3-89564fdf5727 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:06.061299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c171ca-9dc5-4f79-85f3-89564fdf5727 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:06.061585 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c171ca-9dc5-4f79-85f3-89564fdf5727 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:06.062693 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c171ca-9dc5-4f79-85f3-89564fdf5727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:06.072369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c171ca-9dc5-4f79-85f3-89564fdf5727 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:06.087936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1324/2686] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:06.090791 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc310d-6892-4f8a-a997-e4374055671e admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:06.104650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc310d-6892-4f8a-a997-e4374055671e admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:06.109385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98deb3a-c250-4c0a-ae3d-e3f3e401f6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:06.110896 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98deb3a-c250-4c0a-ae3d-e3f3e401f6d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:06.111180 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98deb3a-c250-4c0a-ae3d-e3f3e401f6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:06.111576 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98deb3a-c250-4c0a-ae3d-e3f3e401f6d0 admin admin] PATH_INFO: `/v3/projects/55eb603e4da2438e8ff8a75c656d11f2/users/9e62430e89b34b14abeffc326f7ddf12/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:06.128982 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98deb3a-c250-4c0a-ae3d-e3f3e401f6d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55eb603e4da2438e8ff8a75c656d11f2, user_id=9e62430e89b34b14abeffc326f7ddf12, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:06.132445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98deb3a-c250-4c0a-ae3d-e3f3e401f6d0 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:06.157313 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1363/2687] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:06 2022] PUT /identity/v3/projects/55eb603e4da2438e8ff8a75c656d11f2/users/9e62430e89b34b14abeffc326f7ddf12/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:06.168084 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132d705-233a-4cc8-a7bf-743f13cf6de0 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:06.168455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132d705-233a-4cc8-a7bf-743f13cf6de0 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:06.168743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132d705-233a-4cc8-a7bf-743f13cf6de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:06.228628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-c132d705-233a-4cc8-a7bf-743f13cf6de0 None None] MFA Rules not processed for user `9e62430e89b34b14abeffc326f7ddf12`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:06.228628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c132d705-233a-4cc8-a7bf-743f13cf6de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:06.299600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1325/2688] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:06.304146 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98deb3a-c250-4c0a-ae3d-e3f3e401f6d0 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:06.367665 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98deb3a-c250-4c0a-ae3d-e3f3e401f6d0 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:06.372414 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89c905-b22f-4c1b-bc32-9645a6b39842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:06.373081 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89c905-b22f-4c1b-bc32-9645a6b39842 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:06.373422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89c905-b22f-4c1b-bc32-9645a6b39842 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:06.374183 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89c905-b22f-4c1b-bc32-9645a6b39842 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:06.374895 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89c905-b22f-4c1b-bc32-9645a6b39842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:06.376810 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89c905-b22f-4c1b-bc32-9645a6b39842 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:06.408092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1364/2689] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:34:06 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:34:06.409301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132d705-233a-4cc8-a7bf-743f13cf6de0 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:06.421520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132d705-233a-4cc8-a7bf-743f13cf6de0 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:06.484136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ba240-1587-44e9-8cb9-53e999d5f128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:06.484854 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ba240-1587-44e9-8cb9-53e999d5f128 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:06.485168 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ba240-1587-44e9-8cb9-53e999d5f128 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:06.485464 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ba240-1587-44e9-8cb9-53e999d5f128 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:06.486278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ba240-1587-44e9-8cb9-53e999d5f128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:06.495520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ba240-1587-44e9-8cb9-53e999d5f128 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:06.519406 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1326/2690] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:34:06 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:06.520756 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89c905-b22f-4c1b-bc32-9645a6b39842 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:06.546626 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89c905-b22f-4c1b-bc32-9645a6b39842 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:06.551518 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e84e9-1569-4eae-83f8-d0e247a445f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:06.552466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e84e9-1569-4eae-83f8-d0e247a445f4 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:06.552719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e84e9-1569-4eae-83f8-d0e247a445f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:06.565691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e84e9-1569-4eae-83f8-d0e247a445f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:06.566700 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e84e9-1569-4eae-83f8-d0e247a445f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:06.568748 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e84e9-1569-4eae-83f8-d0e247a445f4 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:06.580508 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1365/2691] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:06.584524 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ba240-1587-44e9-8cb9-53e999d5f128 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:06.594778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ba240-1587-44e9-8cb9-53e999d5f128 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:06.599500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8119a-abb5-4f64-bb82-e209d47555d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:06.600201 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8119a-abb5-4f64-bb82-e209d47555d6 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:06.600488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8119a-abb5-4f64-bb82-e209d47555d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:06.600769 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8119a-abb5-4f64-bb82-e209d47555d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:06.601530 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8119a-abb5-4f64-bb82-e209d47555d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:06.603129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8119a-abb5-4f64-bb82-e209d47555d6 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:06.613170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1327/2692] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:06.617390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e84e9-1569-4eae-83f8-d0e247a445f4 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:06.628341 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e84e9-1569-4eae-83f8-d0e247a445f4 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:06.633072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba580564-7788-4fc9-a3b7-55f197007011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:06.634037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba580564-7788-4fc9-a3b7-55f197007011 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:06.634393 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba580564-7788-4fc9-a3b7-55f197007011 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:06.634741 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba580564-7788-4fc9-a3b7-55f197007011 admin admin] PATH_INFO: `/v3/projects/0534303ec37a4e41a751ccb8ae875b52/users/ddbacb2e63cb4b68973c72048d1b09a8/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:06.650189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba580564-7788-4fc9-a3b7-55f197007011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0534303ec37a4e41a751ccb8ae875b52, user_id=ddbacb2e63cb4b68973c72048d1b09a8, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:06.652881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba580564-7788-4fc9-a3b7-55f197007011 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:06.667280 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1366/2693] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:06 2022] PUT /identity/v3/projects/0534303ec37a4e41a751ccb8ae875b52/users/ddbacb2e63cb4b68973c72048d1b09a8/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:34:06.678942 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2fe3c-1a37-49bf-a985-37a73ed845d1 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:06.679222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2fe3c-1a37-49bf-a985-37a73ed845d1 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:06.679498 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2fe3c-1a37-49bf-a985-37a73ed845d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:06.708974 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-1eb2fe3c-1a37-49bf-a985-37a73ed845d1 None None] MFA Rules not processed for user `ddbacb2e63cb4b68973c72048d1b09a8`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:06.710216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1eb2fe3c-1a37-49bf-a985-37a73ed845d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:06.785335 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1328/2694] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:06.790028 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba580564-7788-4fc9-a3b7-55f197007011 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:06.854192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba580564-7788-4fc9-a3b7-55f197007011 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:06.859495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3bfd3-d50a-4e9f-b555-e3ccf89300f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:06.860206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3bfd3-d50a-4e9f-b555-e3ccf89300f0 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:06.860494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3bfd3-d50a-4e9f-b555-e3ccf89300f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:06.860772 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3bfd3-d50a-4e9f-b555-e3ccf89300f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:06.861481 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3bfd3-d50a-4e9f-b555-e3ccf89300f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:06.863404 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3bfd3-d50a-4e9f-b555-e3ccf89300f0 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:06.894743 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1367/2695] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:34:06 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:06.896221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2fe3c-1a37-49bf-a985-37a73ed845d1 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:06.915270 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2fe3c-1a37-49bf-a985-37a73ed845d1 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:06.982341 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c15d87-0d92-43c9-929a-c41a027f0c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:06.984224 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c15d87-0d92-43c9-929a-c41a027f0c25 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:06.984344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c15d87-0d92-43c9-929a-c41a027f0c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:06.984589 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c15d87-0d92-43c9-929a-c41a027f0c25 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:06.985305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c15d87-0d92-43c9-929a-c41a027f0c25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:06.987332 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c15d87-0d92-43c9-929a-c41a027f0c25 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:07.012795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1329/2696] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:34:06 2022] POST /identity/v3/users => generated 300 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:07.017615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3bfd3-d50a-4e9f-b555-e3ccf89300f0 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:07.027945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3bfd3-d50a-4e9f-b555-e3ccf89300f0 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:07.033691 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ee89b-8321-42f8-a01d-d874cf55fea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:07.034473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ee89b-8321-42f8-a01d-d874cf55fea3 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:07.034843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ee89b-8321-42f8-a01d-d874cf55fea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:07.035182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ee89b-8321-42f8-a01d-d874cf55fea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:07.036003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ee89b-8321-42f8-a01d-d874cf55fea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:07.037668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ee89b-8321-42f8-a01d-d874cf55fea3 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:07.049133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1368/2697] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:34:07.053165 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c15d87-0d92-43c9-929a-c41a027f0c25 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:07.064378 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c15d87-0d92-43c9-929a-c41a027f0c25 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:07.069135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d53a9-1be4-4322-b2fa-0b444e349b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:07.070024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d53a9-1be4-4322-b2fa-0b444e349b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:07.070299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d53a9-1be4-4322-b2fa-0b444e349b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:07.070569 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d53a9-1be4-4322-b2fa-0b444e349b3c admin admin] PATH_INFO: `/v3/projects/cc1f425b749f47cc91cf16d8eba90167/users/e165154c63604ffd955ec35e0b57c464/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:07.087781 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d53a9-1be4-4322-b2fa-0b444e349b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1f425b749f47cc91cf16d8eba90167, user_id=e165154c63604ffd955ec35e0b57c464, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:07.090803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d53a9-1be4-4322-b2fa-0b444e349b3c admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:07.104605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1330/2698] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:07 2022] PUT /identity/v3/projects/cc1f425b749f47cc91cf16d8eba90167/users/e165154c63604ffd955ec35e0b57c464/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:07.108583 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ee89b-8321-42f8-a01d-d874cf55fea3 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:07.118473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ee89b-8321-42f8-a01d-d874cf55fea3 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:07.173708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ad586-b8b0-4969-b6ca-35b1983536ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:07.174306 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad586-b8b0-4969-b6ca-35b1983536ea admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:07.174586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad586-b8b0-4969-b6ca-35b1983536ea admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:07.174889 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad586-b8b0-4969-b6ca-35b1983536ea admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:07.175610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ad586-b8b0-4969-b6ca-35b1983536ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:07.177236 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ad586-b8b0-4969-b6ca-35b1983536ea admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:07.189355 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1369/2699] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:07.193370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d53a9-1be4-4322-b2fa-0b444e349b3c admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:07.205040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d53a9-1be4-4322-b2fa-0b444e349b3c admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:07.210938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b9a97-4bb5-4c3f-945e-f64683e19b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:07.211812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b9a97-4bb5-4c3f-945e-f64683e19b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:07.211812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b9a97-4bb5-4c3f-945e-f64683e19b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:07.212040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b9a97-4bb5-4c3f-945e-f64683e19b87 admin admin] PATH_INFO: `/v3/projects/cc1f425b749f47cc91cf16d8eba90167/users/e165154c63604ffd955ec35e0b57c464/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:07.214053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b9a97-4bb5-4c3f-945e-f64683e19b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1f425b749f47cc91cf16d8eba90167, user_id=e165154c63604ffd955ec35e0b57c464, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:07.217636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b9a97-4bb5-4c3f-945e-f64683e19b87 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:07.236258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1331/2700] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:07 2022] PUT /identity/v3/projects/cc1f425b749f47cc91cf16d8eba90167/users/e165154c63604ffd955ec35e0b57c464/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:07.246434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365c7ad-4880-4faf-b9fa-51a8b2831287 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:07.246727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365c7ad-4880-4faf-b9fa-51a8b2831287 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:07.247007 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365c7ad-4880-4faf-b9fa-51a8b2831287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:07.279706 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c365c7ad-4880-4faf-b9fa-51a8b2831287 None None] MFA Rules not processed for user `e165154c63604ffd955ec35e0b57c464`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:07.281711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c365c7ad-4880-4faf-b9fa-51a8b2831287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:07.355028 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1370/2701] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:07.365984 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b9a97-4bb5-4c3f-945e-f64683e19b87 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:07.367679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-504b9a97-4bb5-4c3f-945e-f64683e19b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:07.423672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b9a97-4bb5-4c3f-945e-f64683e19b87 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:07.427462 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e69f51-7e6d-41aa-9c61-a5d32e8e3e28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:07.428099 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e69f51-7e6d-41aa-9c61-a5d32e8e3e28 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:07.428384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e69f51-7e6d-41aa-9c61-a5d32e8e3e28 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:07.428659 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e69f51-7e6d-41aa-9c61-a5d32e8e3e28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:07.438752 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e69f51-7e6d-41aa-9c61-a5d32e8e3e28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:07.440211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e69f51-7e6d-41aa-9c61-a5d32e8e3e28 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:07.449281 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1332/2702] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:08.038548 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365c7ad-4880-4faf-b9fa-51a8b2831287 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:08.054614 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365c7ad-4880-4faf-b9fa-51a8b2831287 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:08.059232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52ecb9-77a0-4cf4-ab38-f0b711223e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:08.060328 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52ecb9-77a0-4cf4-ab38-f0b711223e22 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:08.061318 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52ecb9-77a0-4cf4-ab38-f0b711223e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:08.061640 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52ecb9-77a0-4cf4-ab38-f0b711223e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:08.127780 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52ecb9-77a0-4cf4-ab38-f0b711223e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:08.129247 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52ecb9-77a0-4cf4-ab38-f0b711223e22 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:08.138847 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1371/2703] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:12.241343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e69f51-7e6d-41aa-9c61-a5d32e8e3e28 service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:12.243812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-47e69f51-7e6d-41aa-9c61-a5d32e8e3e28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:12.255921 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e69f51-7e6d-41aa-9c61-a5d32e8e3e28 service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:12.266904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f44ae-8db8-400f-8575-f4d60f55037a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:12.267572 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f44ae-8db8-400f-8575-f4d60f55037a service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:12.267857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f44ae-8db8-400f-8575-f4d60f55037a service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:12.268170 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f44ae-8db8-400f-8575-f4d60f55037a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:12.338696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f44ae-8db8-400f-8575-f4d60f55037a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:12.340397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f44ae-8db8-400f-8575-f4d60f55037a service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:12.341364 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52ecb9-77a0-4cf4-ab38-f0b711223e22 service cinder] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:12.342805 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-8c52ecb9-77a0-4cf4-ab38-f0b711223e22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:12.351356 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1333/2704] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:12.404307 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52ecb9-77a0-4cf4-ab38-f0b711223e22 service cinder] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:12.408622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a056b7d-a9fb-462d-926e-ab52e47feaca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:12.409263 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a056b7d-a9fb-462d-926e-ab52e47feaca service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:12.409547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a056b7d-a9fb-462d-926e-ab52e47feaca service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:12.409819 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a056b7d-a9fb-462d-926e-ab52e47feaca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:12.411573 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3a056b7d-a9fb-462d-926e-ab52e47feaca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:12.477620 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a056b7d-a9fb-462d-926e-ab52e47feaca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:12.479055 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a056b7d-a9fb-462d-926e-ab52e47feaca service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:12.489932 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1372/2705] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:34:16.740111 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f44ae-8db8-400f-8575-f4d60f55037a service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:16.801776 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f44ae-8db8-400f-8575-f4d60f55037a service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:16.806300 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:16.807103 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:16.807388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:16.807665 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] PATH_INFO: `/v3/users/9e62430e89b34b14abeffc326f7ddf12` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:16.808870 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e62430e89b34b14abeffc326f7ddf12)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:16.810828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:16.855055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e62430e89b34b14abeffc326f7ddf12'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:16.855688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e62430e89b34b14abeffc326f7ddf12'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:16.856301 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e62430e89b34b14abeffc326f7ddf12'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:16.863523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e62430e89b34b14abeffc326f7ddf12'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:16.870084 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e62430e89b34b14abeffc326f7ddf12'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:16.886194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1334/2706] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:34:16 2022] DELETE /identity/v3/users/9e62430e89b34b14abeffc326f7ddf12 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:16.891392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb0057-bfb5-4b0d-b9d8-44bae7da5ac1 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:16.891879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb0057-bfb5-4b0d-b9d8-44bae7da5ac1 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:16.892010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb0057-bfb5-4b0d-b9d8-44bae7da5ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:16.909701 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-4efb0057-bfb5-4b0d-b9d8-44bae7da5ac1 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:16.911131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4efb0057-bfb5-4b0d-b9d8-44bae7da5ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:16.974749 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1373/2707] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:34:16.985065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:16.986580 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:17.053328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78661bc8-3acb-434c-af95-cf4f6a210b47 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:17.057479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6b37a-b9aa-47b8-a750-0a5050bd4ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:17.058431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6b37a-b9aa-47b8-a750-0a5050bd4ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:17.058431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6b37a-b9aa-47b8-a750-0a5050bd4ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:17.058652 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6b37a-b9aa-47b8-a750-0a5050bd4ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:17.069842 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6b37a-b9aa-47b8-a750-0a5050bd4ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:17.071490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6b37a-b9aa-47b8-a750-0a5050bd4ab1 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:17.082288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1335/2708] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:17.653319 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb0057-bfb5-4b0d-b9d8-44bae7da5ac1 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:17.654748 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4efb0057-bfb5-4b0d-b9d8-44bae7da5ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:17.668478 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb0057-bfb5-4b0d-b9d8-44bae7da5ac1 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:17.676274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:17.677434 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:17.677854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:17.678273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] PATH_INFO: `/v3/projects/55eb603e4da2438e8ff8a75c656d11f2` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:17.679652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55eb603e4da2438e8ff8a75c656d11f2)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:17.682060 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:17.741242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55eb603e4da2438e8ff8a75c656d11f2', 'request_id': 'req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:17.750714 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55eb603e4da2438e8ff8a75c656d11f2', 'request_id': 'req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:17.759700 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55eb603e4da2438e8ff8a75c656d11f2', 'request_id': 'req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:17.771170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] The token cache is being invalidate because project 55eb603e4da2438e8ff8a75c656d11f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:34:17.771460 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:17.772274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:17.773832 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1374/2709] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:34:17 2022] DELETE /identity/v3/projects/55eb603e4da2438e8ff8a75c656d11f2 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:17.777808 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6b37a-b9aa-47b8-a750-0a5050bd4ab1 service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:17.790543 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6b37a-b9aa-47b8-a750-0a5050bd4ab1 service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:17.856784 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:17.857791 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:17.858143 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:17.858520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] PATH_INFO: `/v3/users/ddbacb2e63cb4b68973c72048d1b09a8` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:17.860816 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddbacb2e63cb4b68973c72048d1b09a8)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:17.866843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:17.923771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddbacb2e63cb4b68973c72048d1b09a8'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:17.924570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddbacb2e63cb4b68973c72048d1b09a8'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:17.925365 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddbacb2e63cb4b68973c72048d1b09a8'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:17.932710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddbacb2e63cb4b68973c72048d1b09a8'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:17.939695 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddbacb2e63cb4b68973c72048d1b09a8'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:17.956416 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1336/2710] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:34:17 2022] DELETE /identity/v3/users/ddbacb2e63cb4b68973c72048d1b09a8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:34:18.224382 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:18.226079 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:18.248672 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc045e-c12d-4fc0-b2f4-a6f9bcd5e865 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:18.313679 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:18.314734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:18.315017 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:18.315860 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] PATH_INFO: `/v3/projects/0534303ec37a4e41a751ccb8ae875b52` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:18.316529 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0534303ec37a4e41a751ccb8ae875b52)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:18.318437 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:18.369132 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0534303ec37a4e41a751ccb8ae875b52', 'request_id': 'req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.374315 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0534303ec37a4e41a751ccb8ae875b52', 'request_id': 'req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.381330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0534303ec37a4e41a751ccb8ae875b52', 'request_id': 'req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.392436 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] The token cache is being invalidate because project 0534303ec37a4e41a751ccb8ae875b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:34:18.392661 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.393286 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.394812 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1375/2711] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:34:18 2022] DELETE /identity/v3/projects/0534303ec37a4e41a751ccb8ae875b52 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:18.399020 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:18.400468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:18.413333 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd568af-723c-4a78-9c9f-81a41da176fa admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:18.467139 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702182cc-0559-4282-a0d9-b20a03e54ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:18.468157 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702182cc-0559-4282-a0d9-b20a03e54ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:18.468546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702182cc-0559-4282-a0d9-b20a03e54ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:18.468949 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702182cc-0559-4282-a0d9-b20a03e54ff0 admin admin] PATH_INFO: `/v3/users/e165154c63604ffd955ec35e0b57c464` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:18.470217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702182cc-0559-4282-a0d9-b20a03e54ff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e165154c63604ffd955ec35e0b57c464)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:18.472152 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702182cc-0559-4282-a0d9-b20a03e54ff0 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:18.520874 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-702182cc-0559-4282-a0d9-b20a03e54ff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e165154c63604ffd955ec35e0b57c464'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.521529 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-702182cc-0559-4282-a0d9-b20a03e54ff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e165154c63604ffd955ec35e0b57c464'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.522133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-702182cc-0559-4282-a0d9-b20a03e54ff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e165154c63604ffd955ec35e0b57c464'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.529098 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-702182cc-0559-4282-a0d9-b20a03e54ff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e165154c63604ffd955ec35e0b57c464'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.536437 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-702182cc-0559-4282-a0d9-b20a03e54ff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e165154c63604ffd955ec35e0b57c464'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.552850 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1337/2712] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:34:18 2022] DELETE /identity/v3/users/e165154c63604ffd955ec35e0b57c464 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:18.815138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:18.816650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:18.833150 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e8c1-995d-426f-aa91-fb1d3a4ac1c6 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:18.890430 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85590f19-21d8-4d02-aed6-d8f0c70fb16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:18.891285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85590f19-21d8-4d02-aed6-d8f0c70fb16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:18.891567 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85590f19-21d8-4d02-aed6-d8f0c70fb16b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:18.891840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85590f19-21d8-4d02-aed6-d8f0c70fb16b admin admin] PATH_INFO: `/v3/projects/cc1f425b749f47cc91cf16d8eba90167` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:18.893027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85590f19-21d8-4d02-aed6-d8f0c70fb16b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc1f425b749f47cc91cf16d8eba90167)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:18.894959 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85590f19-21d8-4d02-aed6-d8f0c70fb16b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:18.939133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-85590f19-21d8-4d02-aed6-d8f0c70fb16b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc1f425b749f47cc91cf16d8eba90167', 'request_id': 'req-85590f19-21d8-4d02-aed6-d8f0c70fb16b'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.944223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-85590f19-21d8-4d02-aed6-d8f0c70fb16b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc1f425b749f47cc91cf16d8eba90167', 'request_id': 'req-85590f19-21d8-4d02-aed6-d8f0c70fb16b'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.952204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-85590f19-21d8-4d02-aed6-d8f0c70fb16b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc1f425b749f47cc91cf16d8eba90167', 'request_id': 'req-85590f19-21d8-4d02-aed6-d8f0c70fb16b'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.963688 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-85590f19-21d8-4d02-aed6-d8f0c70fb16b admin admin] The token cache is being invalidate because project cc1f425b749f47cc91cf16d8eba90167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:34:18.963999 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-85590f19-21d8-4d02-aed6-d8f0c70fb16b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.964996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-85590f19-21d8-4d02-aed6-d8f0c70fb16b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:18.966509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1376/2713] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:34:18 2022] DELETE /identity/v3/projects/cc1f425b749f47cc91cf16d8eba90167 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:18.972400 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287028db-a431-4106-8435-e81b29b46f6c None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:18.972883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287028db-a431-4106-8435-e81b29b46f6c None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:18.972983 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287028db-a431-4106-8435-e81b29b46f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:18.991081 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-287028db-a431-4106-8435-e81b29b46f6c None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:18.992473 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-287028db-a431-4106-8435-e81b29b46f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:19.051100 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1338/2714] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:19.059565 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1323f50-707d-4ffb-be31-7fb8f5e530ad None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.059855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1323f50-707d-4ffb-be31-7fb8f5e530ad None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.060313 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1323f50-707d-4ffb-be31-7fb8f5e530ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.078596 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-b1323f50-707d-4ffb-be31-7fb8f5e530ad None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:19.079798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b1323f50-707d-4ffb-be31-7fb8f5e530ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:19.087702 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1377/2715] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:19.092351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287028db-a431-4106-8435-e81b29b46f6c None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:19.101975 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287028db-a431-4106-8435-e81b29b46f6c None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:19.106491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee53b8b-b34b-4f84-b793-984061a9f8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:19.107141 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee53b8b-b34b-4f84-b793-984061a9f8fb admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.107423 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee53b8b-b34b-4f84-b793-984061a9f8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.107710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee53b8b-b34b-4f84-b793-984061a9f8fb admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.108938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee53b8b-b34b-4f84-b793-984061a9f8fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:34:19.109347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee53b8b-b34b-4f84-b793-984061a9f8fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:19.110935 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee53b8b-b34b-4f84-b793-984061a9f8fb admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:19.121866 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1339/2716] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:34:19.126840 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1323f50-707d-4ffb-be31-7fb8f5e530ad None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:19.135977 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1323f50-707d-4ffb-be31-7fb8f5e530ad None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:19.140904 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd4042-b843-425a-b860-6338c25c193b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:19.141724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd4042-b843-425a-b860-6338c25c193b admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.142008 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd4042-b843-425a-b860-6338c25c193b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.142330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd4042-b843-425a-b860-6338c25c193b admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.143043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd4042-b843-425a-b860-6338c25c193b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:19.145036 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd4042-b843-425a-b860-6338c25c193b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:19.169495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1378/2717] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:34:19 2022] POST /identity/v3/projects => generated 383 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:19.174522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee53b8b-b34b-4f84-b793-984061a9f8fb admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:19.185124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee53b8b-b34b-4f84-b793-984061a9f8fb admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:19.241640 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8557dfe-883f-4d61-847e-fac8c3f1747a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:19.242314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8557dfe-883f-4d61-847e-fac8c3f1747a admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.242600 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8557dfe-883f-4d61-847e-fac8c3f1747a admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.242881 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8557dfe-883f-4d61-847e-fac8c3f1747a admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.243583 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8557dfe-883f-4d61-847e-fac8c3f1747a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:19.245516 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8557dfe-883f-4d61-847e-fac8c3f1747a admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:19.267175 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1340/2718] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:34:19 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:19.268492 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd4042-b843-425a-b860-6338c25c193b admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:19.278713 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd4042-b843-425a-b860-6338c25c193b admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:19.284000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851e84b-653f-4d8a-aca4-a5501e2b75fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:19.286464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851e84b-653f-4d8a-aca4-a5501e2b75fc admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.286798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851e84b-653f-4d8a-aca4-a5501e2b75fc admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.287116 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851e84b-653f-4d8a-aca4-a5501e2b75fc admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.287841 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851e84b-653f-4d8a-aca4-a5501e2b75fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:19.289454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851e84b-653f-4d8a-aca4-a5501e2b75fc admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:19.300340 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1379/2719] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:19.304366 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8557dfe-883f-4d61-847e-fac8c3f1747a admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:19.314319 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8557dfe-883f-4d61-847e-fac8c3f1747a admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:19.318803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65415d-670d-4821-808f-c357bf477891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:19.319453 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65415d-670d-4821-808f-c357bf477891 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.319734 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65415d-670d-4821-808f-c357bf477891 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.320051 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65415d-670d-4821-808f-c357bf477891 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.320774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65415d-670d-4821-808f-c357bf477891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:19.322331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65415d-670d-4821-808f-c357bf477891 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:19.332084 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1341/2720] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:19.336264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851e84b-653f-4d8a-aca4-a5501e2b75fc admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:19.346668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851e84b-653f-4d8a-aca4-a5501e2b75fc admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:19.351190 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780333e4-bc48-43d0-a51b-7e1f90f00279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:19.352095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780333e4-bc48-43d0-a51b-7e1f90f00279 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.352376 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780333e4-bc48-43d0-a51b-7e1f90f00279 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.352655 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780333e4-bc48-43d0-a51b-7e1f90f00279 admin admin] PATH_INFO: `/v3/projects/3ec9612e58314db0984f080da0772ad2/users/ec83e8f0c6944d0084cb762a049e9b6a/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.367287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780333e4-bc48-43d0-a51b-7e1f90f00279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec9612e58314db0984f080da0772ad2, user_id=ec83e8f0c6944d0084cb762a049e9b6a, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:19.369957 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780333e4-bc48-43d0-a51b-7e1f90f00279 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:19.383527 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1380/2721] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:19 2022] PUT /identity/v3/projects/3ec9612e58314db0984f080da0772ad2/users/ec83e8f0c6944d0084cb762a049e9b6a/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:19.395921 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d675ea1-7daf-4a07-9e55-9d759fa5effb None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.396148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d675ea1-7daf-4a07-9e55-9d759fa5effb None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.396428 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d675ea1-7daf-4a07-9e55-9d759fa5effb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.424119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-4d675ea1-7daf-4a07-9e55-9d759fa5effb None None] MFA Rules not processed for user `ec83e8f0c6944d0084cb762a049e9b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:19.425315 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-4d675ea1-7daf-4a07-9e55-9d759fa5effb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:19.488254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1342/2722] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:19.492927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780333e4-bc48-43d0-a51b-7e1f90f00279 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:19.552659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780333e4-bc48-43d0-a51b-7e1f90f00279 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:19.557212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cacb68-6a93-49fe-90ca-068954f7419d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:19.557859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cacb68-6a93-49fe-90ca-068954f7419d admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.558137 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cacb68-6a93-49fe-90ca-068954f7419d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.558426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cacb68-6a93-49fe-90ca-068954f7419d admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.559089 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cacb68-6a93-49fe-90ca-068954f7419d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:19.560997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cacb68-6a93-49fe-90ca-068954f7419d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:19.584555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1381/2723] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:34:19 2022] POST /identity/v3/projects => generated 383 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:19.588735 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d675ea1-7daf-4a07-9e55-9d759fa5effb None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:19.600363 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d675ea1-7daf-4a07-9e55-9d759fa5effb None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:19.653436 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24ed07-9741-486a-9a74-490f8c4f8422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:19.654015 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24ed07-9741-486a-9a74-490f8c4f8422 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.654295 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24ed07-9741-486a-9a74-490f8c4f8422 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.654567 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24ed07-9741-486a-9a74-490f8c4f8422 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.655233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24ed07-9741-486a-9a74-490f8c4f8422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:19.657722 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24ed07-9741-486a-9a74-490f8c4f8422 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:19.680231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1343/2724] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:34:19 2022] POST /identity/v3/users => generated 298 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:19.681399 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cacb68-6a93-49fe-90ca-068954f7419d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:19.691275 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cacb68-6a93-49fe-90ca-068954f7419d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:19.695834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af1f0c-33dd-4aaf-9f54-9a417fb9af6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:19.696522 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af1f0c-33dd-4aaf-9f54-9a417fb9af6b admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.696801 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af1f0c-33dd-4aaf-9f54-9a417fb9af6b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.697071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af1f0c-33dd-4aaf-9f54-9a417fb9af6b admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.697792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af1f0c-33dd-4aaf-9f54-9a417fb9af6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:19.699374 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af1f0c-33dd-4aaf-9f54-9a417fb9af6b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:19.711438 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1382/2725] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:19.715952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24ed07-9741-486a-9a74-490f8c4f8422 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:19.726275 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24ed07-9741-486a-9a74-490f8c4f8422 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:19.730872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff8273-fc87-4a5e-ac09-1c71b181565c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:19.731737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff8273-fc87-4a5e-ac09-1c71b181565c admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.732053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff8273-fc87-4a5e-ac09-1c71b181565c admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.732330 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff8273-fc87-4a5e-ac09-1c71b181565c admin admin] PATH_INFO: `/v3/projects/2d82eb437df4410388c415d158d5285e/users/ac75af997f8047958705cc831ac66f30/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.746726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff8273-fc87-4a5e-ac09-1c71b181565c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d82eb437df4410388c415d158d5285e, user_id=ac75af997f8047958705cc831ac66f30, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:19.749498 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff8273-fc87-4a5e-ac09-1c71b181565c admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:19.763119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1344/2726] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:19 2022] PUT /identity/v3/projects/2d82eb437df4410388c415d158d5285e/users/ac75af997f8047958705cc831ac66f30/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:34:19.767722 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af1f0c-33dd-4aaf-9f54-9a417fb9af6b admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:19.777348 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af1f0c-33dd-4aaf-9f54-9a417fb9af6b admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:19.830466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174186c4-8520-49b5-839f-663d6d9081bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:19.831131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174186c4-8520-49b5-839f-663d6d9081bf admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.831413 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174186c4-8520-49b5-839f-663d6d9081bf admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.831712 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174186c4-8520-49b5-839f-663d6d9081bf admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.832467 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174186c4-8520-49b5-839f-663d6d9081bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:19.834047 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174186c4-8520-49b5-839f-663d6d9081bf admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:19.844549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1383/2727] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:19.848624 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff8273-fc87-4a5e-ac09-1c71b181565c admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:19.858500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff8273-fc87-4a5e-ac09-1c71b181565c admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:19.863064 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c36b2e-0f9a-4845-9425-d370c280a320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:19.863960 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c36b2e-0f9a-4845-9425-d370c280a320 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.864245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c36b2e-0f9a-4845-9425-d370c280a320 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.864520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c36b2e-0f9a-4845-9425-d370c280a320 admin admin] PATH_INFO: `/v3/projects/2d82eb437df4410388c415d158d5285e/users/ac75af997f8047958705cc831ac66f30/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.866434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c36b2e-0f9a-4845-9425-d370c280a320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d82eb437df4410388c415d158d5285e, user_id=ac75af997f8047958705cc831ac66f30, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:19.869056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c36b2e-0f9a-4845-9425-d370c280a320 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:19.878722 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1345/2728] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:19 2022] PUT /identity/v3/projects/2d82eb437df4410388c415d158d5285e/users/ac75af997f8047958705cc831ac66f30/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:34:19.888559 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ac32d-5b11-40b8-a5a5-253e8548d3a1 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:19.888853 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ac32d-5b11-40b8-a5a5-253e8548d3a1 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:19.889133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ac32d-5b11-40b8-a5a5-253e8548d3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:19.915960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-df8ac32d-5b11-40b8-a5a5-253e8548d3a1 None None] MFA Rules not processed for user `ac75af997f8047958705cc831ac66f30`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:19.917409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-df8ac32d-5b11-40b8-a5a5-253e8548d3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:19.983173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1384/2729] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:19.994939 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c36b2e-0f9a-4845-9425-d370c280a320 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:19.996707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a3c36b2e-0f9a-4845-9425-d370c280a320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:20.052688 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c36b2e-0f9a-4845-9425-d370c280a320 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:20.056296 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a95f-4db2-4a2d-9d8c-82cfe7e4f654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:20.056938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a95f-4db2-4a2d-9d8c-82cfe7e4f654 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:20.057222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a95f-4db2-4a2d-9d8c-82cfe7e4f654 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:20.057497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4a95f-4db2-4a2d-9d8c-82cfe7e4f654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:20.066564 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4a95f-4db2-4a2d-9d8c-82cfe7e4f654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:20.067999 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4a95f-4db2-4a2d-9d8c-82cfe7e4f654 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:20.077072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1346/2730] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:20.163526 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ac32d-5b11-40b8-a5a5-253e8548d3a1 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:20.173362 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ac32d-5b11-40b8-a5a5-253e8548d3a1 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:20.176969 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab084c10-d103-436e-b2b3-2e03772da0bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:20.177612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab084c10-d103-436e-b2b3-2e03772da0bd service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:20.177894 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab084c10-d103-436e-b2b3-2e03772da0bd service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:20.178171 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab084c10-d103-436e-b2b3-2e03772da0bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:20.235743 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab084c10-d103-436e-b2b3-2e03772da0bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:20.237206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab084c10-d103-436e-b2b3-2e03772da0bd service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:20.245974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1385/2731] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:20.501744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a95f-4db2-4a2d-9d8c-82cfe7e4f654 service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:20.562563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4a95f-4db2-4a2d-9d8c-82cfe7e4f654 service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:20.567370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:20.568282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:20.568566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:20.568897 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] PATH_INFO: `/v3/users/ec83e8f0c6944d0084cb762a049e9b6a` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:20.570136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec83e8f0c6944d0084cb762a049e9b6a)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:20.572184 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:20.632048 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec83e8f0c6944d0084cb762a049e9b6a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:20.632898 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec83e8f0c6944d0084cb762a049e9b6a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:20.633648 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec83e8f0c6944d0084cb762a049e9b6a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:20.642091 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec83e8f0c6944d0084cb762a049e9b6a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:20.650015 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec83e8f0c6944d0084cb762a049e9b6a'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:20.669729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1347/2732] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:34:20 2022] DELETE /identity/v3/users/ec83e8f0c6944d0084cb762a049e9b6a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:20.675160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d87d9-1d39-4b11-a4f6-b1ad33f408d1 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:20.675633 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d87d9-1d39-4b11-a4f6-b1ad33f408d1 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:20.675633 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d87d9-1d39-4b11-a4f6-b1ad33f408d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:20.693130 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-946d87d9-1d39-4b11-a4f6-b1ad33f408d1 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:20.694330 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-946d87d9-1d39-4b11-a4f6-b1ad33f408d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:20.755320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1386/2733] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:20.766144 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:20.767562 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:20.825876 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99b7f3-9224-475f-88a4-882c44cef0cb admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:20.829571 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dddd5-45cc-4192-a29e-c7edb09f1f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:20.830231 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dddd5-45cc-4192-a29e-c7edb09f1f4d service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:20.830501 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dddd5-45cc-4192-a29e-c7edb09f1f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:20.830783 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dddd5-45cc-4192-a29e-c7edb09f1f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:20.839727 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dddd5-45cc-4192-a29e-c7edb09f1f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:20.841196 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dddd5-45cc-4192-a29e-c7edb09f1f4d service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:20.851312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1348/2734] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:21.048444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d87d9-1d39-4b11-a4f6-b1ad33f408d1 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:21.049954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-946d87d9-1d39-4b11-a4f6-b1ad33f408d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:21.062422 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d87d9-1d39-4b11-a4f6-b1ad33f408d1 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:21.068704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0e1c1-f8d2-44e2-9f56-67eb7f9330ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:21.070271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0e1c1-f8d2-44e2-9f56-67eb7f9330ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:21.070555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0e1c1-f8d2-44e2-9f56-67eb7f9330ef admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:21.070834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0e1c1-f8d2-44e2-9f56-67eb7f9330ef admin admin] PATH_INFO: `/v3/users/a7be5a11e4d2458897ece6b9b04fded9` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:21.072391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0e1c1-f8d2-44e2-9f56-67eb7f9330ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7be5a11e4d2458897ece6b9b04fded9)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:21.074340 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0e1c1-f8d2-44e2-9f56-67eb7f9330ef admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:21.124944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2bb0e1c1-f8d2-44e2-9f56-67eb7f9330ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7be5a11e4d2458897ece6b9b04fded9'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.124944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2bb0e1c1-f8d2-44e2-9f56-67eb7f9330ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7be5a11e4d2458897ece6b9b04fded9'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.126446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dddd5-45cc-4192-a29e-c7edb09f1f4d service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:21.128239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-730dddd5-45cc-4192-a29e-c7edb09f1f4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:21.134096 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2bb0e1c1-f8d2-44e2-9f56-67eb7f9330ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7be5a11e4d2458897ece6b9b04fded9'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.143606 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2bb0e1c1-f8d2-44e2-9f56-67eb7f9330ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7be5a11e4d2458897ece6b9b04fded9'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.144332 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2bb0e1c1-f8d2-44e2-9f56-67eb7f9330ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7be5a11e4d2458897ece6b9b04fded9'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.152104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dddd5-45cc-4192-a29e-c7edb09f1f4d service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:21.159449 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:21.160470 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:21.160913 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:21.161271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] PATH_INFO: `/v3/projects/3ec9612e58314db0984f080da0772ad2` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:21.162531 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec9612e58314db0984f080da0772ad2)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:21.164563 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:21.165212 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1387/2735] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:34:21 2022] DELETE /identity/v3/users/a7be5a11e4d2458897ece6b9b04fded9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:21.169531 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d0d16-58e6-44aa-b0e9-2f00602effde None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:21.169815 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d0d16-58e6-44aa-b0e9-2f00602effde None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:21.170094 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d0d16-58e6-44aa-b0e9-2f00602effde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:21.186675 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-782d0d16-58e6-44aa-b0e9-2f00602effde None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:21.188127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-782d0d16-58e6-44aa-b0e9-2f00602effde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:21.213199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec9612e58314db0984f080da0772ad2', 'request_id': 'req-297f7f36-5595-468d-bcd2-160e8f5d2a86'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.220085 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec9612e58314db0984f080da0772ad2', 'request_id': 'req-297f7f36-5595-468d-bcd2-160e8f5d2a86'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.225441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec9612e58314db0984f080da0772ad2', 'request_id': 'req-297f7f36-5595-468d-bcd2-160e8f5d2a86'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.239459 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] The token cache is being invalidate because project 3ec9612e58314db0984f080da0772ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:34:21.239750 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.240423 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.242171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1349/2736] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:34:21 2022] DELETE /identity/v3/projects/3ec9612e58314db0984f080da0772ad2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:34:21.246314 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:21.247850 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:21.250208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1388/2737] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:34:21.271109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d0d16-58e6-44aa-b0e9-2f00602effde None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:21.273930 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-782d0d16-58e6-44aa-b0e9-2f00602effde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:21.320302 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f7f36-5595-468d-bcd2-160e8f5d2a86 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:21.325186 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:21.326129 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:21.326405 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:21.326679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] PATH_INFO: `/v3/users/ac75af997f8047958705cc831ac66f30` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:21.328449 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac75af997f8047958705cc831ac66f30)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:21.330786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:21.340229 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d0d16-58e6-44aa-b0e9-2f00602effde None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:21.389014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac75af997f8047958705cc831ac66f30'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.390717 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac75af997f8047958705cc831ac66f30'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.391124 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac75af997f8047958705cc831ac66f30'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.402617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac75af997f8047958705cc831ac66f30'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.404876 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8b27a-1270-404d-84b6-02b2fed4bf69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:21.405813 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8b27a-1270-404d-84b6-02b2fed4bf69 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:21.406135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8b27a-1270-404d-84b6-02b2fed4bf69 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:21.406473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8b27a-1270-404d-84b6-02b2fed4bf69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:21.408673 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b2f8b27a-1270-404d-84b6-02b2fed4bf69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:21.413956 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac75af997f8047958705cc831ac66f30'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.436107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8b27a-1270-404d-84b6-02b2fed4bf69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:21.437623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8b27a-1270-404d-84b6-02b2fed4bf69 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:21.440786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1350/2738] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:34:21 2022] DELETE /identity/v3/users/ac75af997f8047958705cc831ac66f30 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:21.455727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1389/2739] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:21.734206 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:21.736053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:21.832666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62245bb3-d2a3-45ba-8778-fcc197012f23 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:21.837331 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:21.838198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:21.838484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:21.838754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3 admin admin] PATH_INFO: `/v3/projects/2d82eb437df4410388c415d158d5285e` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:21.840347 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d82eb437df4410388c415d158d5285e)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:21.842240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:21.901695 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d82eb437df4410388c415d158d5285e', 'request_id': 'req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.909255 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8b27a-1270-404d-84b6-02b2fed4bf69 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:21.910773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b2f8b27a-1270-404d-84b6-02b2fed4bf69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:21.914749 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d82eb437df4410388c415d158d5285e', 'request_id': 'req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.923484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d82eb437df4410388c415d158d5285e', 'request_id': 'req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.936875 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8b27a-1270-404d-84b6-02b2fed4bf69 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:21.941309 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3 admin admin] The token cache is being invalidate because project 2d82eb437df4410388c415d158d5285e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:34:21.941811 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.943222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c91d4b93-8fea-4378-bf3d-f0d727ed0fc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:21.946321 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1351/2740] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:34:21 2022] DELETE /identity/v3/projects/2d82eb437df4410388c415d158d5285e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:21.954147 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64f279-37b9-4516-8f4b-f6eb2cc1c0f6 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:21.954641 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64f279-37b9-4516-8f4b-f6eb2cc1c0f6 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:21.955105 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64f279-37b9-4516-8f4b-f6eb2cc1c0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:21.978912 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-5f64f279-37b9-4516-8f4b-f6eb2cc1c0f6 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:21.980454 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5f64f279-37b9-4516-8f4b-f6eb2cc1c0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:21.999830 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1ea58-41b3-4874-bcc6-affa87f305f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.000780 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1ea58-41b3-4874-bcc6-affa87f305f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.001067 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1ea58-41b3-4874-bcc6-affa87f305f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.001344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1ea58-41b3-4874-bcc6-affa87f305f3 admin admin] PATH_INFO: `/v3/projects/b765dad498034654800d9a668398a296` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.003098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1ea58-41b3-4874-bcc6-affa87f305f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b765dad498034654800d9a668398a296)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.005040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1ea58-41b3-4874-bcc6-affa87f305f3 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.043375 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1352/2741] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:22.050411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dc1ea58-41b3-4874-bcc6-affa87f305f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b765dad498034654800d9a668398a296', 'request_id': 'req-4dc1ea58-41b3-4874-bcc6-affa87f305f3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:22.052323 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60695a2e-66fe-437f-b3bc-f7cdd46fc0ca None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.052611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60695a2e-66fe-437f-b3bc-f7cdd46fc0ca None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.052890 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60695a2e-66fe-437f-b3bc-f7cdd46fc0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.055746 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dc1ea58-41b3-4874-bcc6-affa87f305f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b765dad498034654800d9a668398a296', 'request_id': 'req-4dc1ea58-41b3-4874-bcc6-affa87f305f3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:22.064526 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dc1ea58-41b3-4874-bcc6-affa87f305f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b765dad498034654800d9a668398a296', 'request_id': 'req-4dc1ea58-41b3-4874-bcc6-affa87f305f3'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:22.071463 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-60695a2e-66fe-437f-b3bc-f7cdd46fc0ca None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:22.072790 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-60695a2e-66fe-437f-b3bc-f7cdd46fc0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:22.075972 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dc1ea58-41b3-4874-bcc6-affa87f305f3 admin admin] The token cache is being invalidate because project b765dad498034654800d9a668398a296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:34:22.076286 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dc1ea58-41b3-4874-bcc6-affa87f305f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:22.077232 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-4dc1ea58-41b3-4874-bcc6-affa87f305f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:22.079424 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1390/2742] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:34:21 2022] DELETE /identity/v3/projects/b765dad498034654800d9a668398a296 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:22.084220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629d586-2f08-423a-bc9a-22ef1e24ea7e None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.084510 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629d586-2f08-423a-bc9a-22ef1e24ea7e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.084792 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629d586-2f08-423a-bc9a-22ef1e24ea7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.113431 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-7629d586-2f08-423a-bc9a-22ef1e24ea7e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:22.114780 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-7629d586-2f08-423a-bc9a-22ef1e24ea7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:22.136744 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1353/2743] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:34:22.141328 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60695a2e-66fe-437f-b3bc-f7cdd46fc0ca None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.150060 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60695a2e-66fe-437f-b3bc-f7cdd46fc0ca None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.154704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8442d-3060-4990-99fa-085d376e2936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.155396 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8442d-3060-4990-99fa-085d376e2936 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.155761 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8442d-3060-4990-99fa-085d376e2936 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.156072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8442d-3060-4990-99fa-085d376e2936 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.156931 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8442d-3060-4990-99fa-085d376e2936 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:34:22.157240 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8442d-3060-4990-99fa-085d376e2936 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.158841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8442d-3060-4990-99fa-085d376e2936 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.173998 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1354/2744] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:34:22.175041 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8442d-3060-4990-99fa-085d376e2936 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.180411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1391/2745] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:34:22.184787 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8442d-3060-4990-99fa-085d376e2936 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.190219 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57a9ac-93b0-4305-a4e5-f1677196879d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.191546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57a9ac-93b0-4305-a4e5-f1677196879d admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.191830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57a9ac-93b0-4305-a4e5-f1677196879d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.191971 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec10aab-a397-4641-8daa-6e809538a93f None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.192189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec10aab-a397-4641-8daa-6e809538a93f None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.192288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57a9ac-93b0-4305-a4e5-f1677196879d admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.192473 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec10aab-a397-4641-8daa-6e809538a93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.192884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57a9ac-93b0-4305-a4e5-f1677196879d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.194771 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce57a9ac-93b0-4305-a4e5-f1677196879d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.210206 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-cec10aab-a397-4641-8daa-6e809538a93f None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:22.211692 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cec10aab-a397-4641-8daa-6e809538a93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:22.225135 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1355/2746] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:34:22 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:34:22.225312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1392/2747] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:22.226131 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57a9ac-93b0-4305-a4e5-f1677196879d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.226247 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec10aab-a397-4641-8daa-6e809538a93f None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.236463 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec10aab-a397-4641-8daa-6e809538a93f None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.287499 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57a9ac-93b0-4305-a4e5-f1677196879d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.291004 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6037ba-1657-42a2-94c0-a523d9ba029b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.291885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6037ba-1657-42a2-94c0-a523d9ba029b admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.292066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399935d2-005d-4fd7-bd7c-f5165e32c072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.292182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6037ba-1657-42a2-94c0-a523d9ba029b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.292459 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6037ba-1657-42a2-94c0-a523d9ba029b admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.292721 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399935d2-005d-4fd7-bd7c-f5165e32c072 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.293001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399935d2-005d-4fd7-bd7c-f5165e32c072 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.293188 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6037ba-1657-42a2-94c0-a523d9ba029b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:34:22.293288 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399935d2-005d-4fd7-bd7c-f5165e32c072 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.293494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6037ba-1657-42a2-94c0-a523d9ba029b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.293933 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399935d2-005d-4fd7-bd7c-f5165e32c072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.295203 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6037ba-1657-42a2-94c0-a523d9ba029b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.295814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399935d2-005d-4fd7-bd7c-f5165e32c072 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.307498 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1393/2748] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:34:22.311979 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6037ba-1657-42a2-94c0-a523d9ba029b admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.316786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1356/2749] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:34:22 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:22.320775 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399935d2-005d-4fd7-bd7c-f5165e32c072 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.322072 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6037ba-1657-42a2-94c0-a523d9ba029b admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.326704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6923605-65b7-4513-b16b-6023b1ea5a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.327392 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6923605-65b7-4513-b16b-6023b1ea5a74 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.327663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6923605-65b7-4513-b16b-6023b1ea5a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.327970 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6923605-65b7-4513-b16b-6023b1ea5a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.328666 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6923605-65b7-4513-b16b-6023b1ea5a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.330417 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399935d2-005d-4fd7-bd7c-f5165e32c072 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.330588 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6923605-65b7-4513-b16b-6023b1ea5a74 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.335018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af805584-ea60-4b80-9bbc-dfde3373b611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.335679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af805584-ea60-4b80-9bbc-dfde3373b611 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.335989 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af805584-ea60-4b80-9bbc-dfde3373b611 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.336264 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af805584-ea60-4b80-9bbc-dfde3373b611 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.336973 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af805584-ea60-4b80-9bbc-dfde3373b611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.338513 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af805584-ea60-4b80-9bbc-dfde3373b611 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.349239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1357/2750] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:34:22.353195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af805584-ea60-4b80-9bbc-dfde3373b611 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.357768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1394/2751] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:34:22 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:34:22.362760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6923605-65b7-4513-b16b-6023b1ea5a74 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.363322 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af805584-ea60-4b80-9bbc-dfde3373b611 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.418710 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1bddb-85dd-4c9b-8107-0ec9add694e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.419383 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1bddb-85dd-4c9b-8107-0ec9add694e9 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.419669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1bddb-85dd-4c9b-8107-0ec9add694e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.419970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1bddb-85dd-4c9b-8107-0ec9add694e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.420410 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6923605-65b7-4513-b16b-6023b1ea5a74 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.420702 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1bddb-85dd-4c9b-8107-0ec9add694e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.422344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1bddb-85dd-4c9b-8107-0ec9add694e9 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.426173 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb865e-5bea-4d17-a208-d51b350332e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.426846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb865e-5bea-4d17-a208-d51b350332e9 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.427133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb865e-5bea-4d17-a208-d51b350332e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.427412 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb865e-5bea-4d17-a208-d51b350332e9 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.428140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb865e-5bea-4d17-a208-d51b350332e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.430135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb865e-5bea-4d17-a208-d51b350332e9 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.439956 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1358/2752] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:34:22.441236 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1bddb-85dd-4c9b-8107-0ec9add694e9 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.451581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1bddb-85dd-4c9b-8107-0ec9add694e9 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.453111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1395/2753] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:34:22 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:34:22.457023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb865e-5bea-4d17-a208-d51b350332e9 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.459955 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b8b87-8fe8-43aa-9519-af7d3d5d938e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.460843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b8b87-8fe8-43aa-9519-af7d3d5d938e admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.461191 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b8b87-8fe8-43aa-9519-af7d3d5d938e admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.461480 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b8b87-8fe8-43aa-9519-af7d3d5d938e admin admin] PATH_INFO: `/v3/projects/20b20f5add5d405fb3c51185fab9c35b/users/12b81c49a65e4059b52a4abe2966933a/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.470498 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb865e-5bea-4d17-a208-d51b350332e9 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.475982 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7550e-d5e3-4d63-99e3-26f2fe66ff61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.476653 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7550e-d5e3-4d63-99e3-26f2fe66ff61 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.476937 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7550e-d5e3-4d63-99e3-26f2fe66ff61 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.477146 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b8b87-8fe8-43aa-9519-af7d3d5d938e admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b20f5add5d405fb3c51185fab9c35b, user_id=12b81c49a65e4059b52a4abe2966933a, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.477273 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7550e-d5e3-4d63-99e3-26f2fe66ff61 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.477933 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7550e-d5e3-4d63-99e3-26f2fe66ff61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.479668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7550e-d5e3-4d63-99e3-26f2fe66ff61 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.479780 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b8b87-8fe8-43aa-9519-af7d3d5d938e admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.493704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1396/2754] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:34:22.494115 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1359/2755] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:22 2022] PUT /identity/v3/projects/20b20f5add5d405fb3c51185fab9c35b/users/12b81c49a65e4059b52a4abe2966933a/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:22.494684 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7550e-d5e3-4d63-99e3-26f2fe66ff61 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.498602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f617eb-6dad-45e8-9442-bdf4504d0032 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.498884 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f617eb-6dad-45e8-9442-bdf4504d0032 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.499663 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f617eb-6dad-45e8-9442-bdf4504d0032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.505505 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7550e-d5e3-4d63-99e3-26f2fe66ff61 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.516678 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-a5f617eb-6dad-45e8-9442-bdf4504d0032 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:22.517938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a5f617eb-6dad-45e8-9442-bdf4504d0032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:22.564391 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c4799-cb63-46fa-b70f-467a639c035f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.565059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c4799-cb63-46fa-b70f-467a639c035f admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.565343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c4799-cb63-46fa-b70f-467a639c035f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.565619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c4799-cb63-46fa-b70f-467a639c035f admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.566445 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c4799-cb63-46fa-b70f-467a639c035f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.568103 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c4799-cb63-46fa-b70f-467a639c035f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.579370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1397/2756] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:34:22.583549 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c4799-cb63-46fa-b70f-467a639c035f admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.587882 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1360/2757] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:34:22.593426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c4799-cb63-46fa-b70f-467a639c035f admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.597841 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f617eb-6dad-45e8-9442-bdf4504d0032 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.598219 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104da0cd-33ff-411b-879e-60c47e8ad1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.599102 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104da0cd-33ff-411b-879e-60c47e8ad1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.599214 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a5f617eb-6dad-45e8-9442-bdf4504d0032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:22.599385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104da0cd-33ff-411b-879e-60c47e8ad1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.599663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104da0cd-33ff-411b-879e-60c47e8ad1f3 admin admin] PATH_INFO: `/v3/projects/682bf69acd354afd832f1b951ded2afe/users/309ce30960ff493795d913821c558c70/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.614901 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104da0cd-33ff-411b-879e-60c47e8ad1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=682bf69acd354afd832f1b951ded2afe, user_id=309ce30960ff493795d913821c558c70, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.617584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104da0cd-33ff-411b-879e-60c47e8ad1f3 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.630871 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1398/2758] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:22 2022] PUT /identity/v3/projects/682bf69acd354afd832f1b951ded2afe/users/309ce30960ff493795d913821c558c70/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:34:22.641310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c1aad-6d33-42a9-81e4-d94f3ecc1787 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.641599 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c1aad-6d33-42a9-81e4-d94f3ecc1787 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.641885 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c1aad-6d33-42a9-81e4-d94f3ecc1787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.656782 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f617eb-6dad-45e8-9442-bdf4504d0032 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.666908 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-e89c1aad-6d33-42a9-81e4-d94f3ecc1787 None None] MFA Rules not processed for user `309ce30960ff493795d913821c558c70`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:22.668122 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-e89c1aad-6d33-42a9-81e4-d94f3ecc1787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:22.704065 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076f444-5c56-4f0c-83fb-0000e1ceca0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.704704 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076f444-5c56-4f0c-83fb-0000e1ceca0d service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.704985 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076f444-5c56-4f0c-83fb-0000e1ceca0d service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.705263 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076f444-5c56-4f0c-83fb-0000e1ceca0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.731288 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1399/2759] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:34:22.735724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c1aad-6d33-42a9-81e4-d94f3ecc1787 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.765754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076f444-5c56-4f0c-83fb-0000e1ceca0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.767282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076f444-5c56-4f0c-83fb-0000e1ceca0d service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.786090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1361/2760] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 04:34:22.803332 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c1aad-6d33-42a9-81e4-d94f3ecc1787 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.808093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c8058-7f62-4814-aca5-0edf06d96ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.808761 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c8058-7f62-4814-aca5-0edf06d96ace admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.809041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c8058-7f62-4814-aca5-0edf06d96ace admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.809312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c8058-7f62-4814-aca5-0edf06d96ace admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.810005 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c8058-7f62-4814-aca5-0edf06d96ace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.812095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c8058-7f62-4814-aca5-0edf06d96ace admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.838350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1400/2761] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:34:22 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:22.842325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076f444-5c56-4f0c-83fb-0000e1ceca0d service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.852581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076f444-5c56-4f0c-83fb-0000e1ceca0d service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.917754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29036c89-f719-45db-9274-c2710d42231e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.918623 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29036c89-f719-45db-9274-c2710d42231e admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.919179 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29036c89-f719-45db-9274-c2710d42231e admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.919626 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29036c89-f719-45db-9274-c2710d42231e admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.920510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29036c89-f719-45db-9274-c2710d42231e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.922726 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29036c89-f719-45db-9274-c2710d42231e admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.944211 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1362/2762] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:34:22 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:22.948187 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c8058-7f62-4814-aca5-0edf06d96ace admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.961615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c8058-7f62-4814-aca5-0edf06d96ace admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:22.967039 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a74475-3b71-44c7-b3b3-f9bf440ab013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:22.968446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a74475-3b71-44c7-b3b3-f9bf440ab013 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:22.968798 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a74475-3b71-44c7-b3b3-f9bf440ab013 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:22.969140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a74475-3b71-44c7-b3b3-f9bf440ab013 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:22.970186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a74475-3b71-44c7-b3b3-f9bf440ab013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:22.971850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a74475-3b71-44c7-b3b3-f9bf440ab013 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:22.982645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1401/2763] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:22.987479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29036c89-f719-45db-9274-c2710d42231e admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:22.997700 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29036c89-f719-45db-9274-c2710d42231e admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:23.002566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7688af9-baef-4398-8c21-dac850e31c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:23.003450 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7688af9-baef-4398-8c21-dac850e31c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:23.003729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7688af9-baef-4398-8c21-dac850e31c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:23.004030 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7688af9-baef-4398-8c21-dac850e31c49 admin admin] PATH_INFO: `/v3/projects/0fe77dd73eaa4e97af1282dbcf700410/users/716402a2fc9244b8847e832aea2deb77/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:23.019796 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7688af9-baef-4398-8c21-dac850e31c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe77dd73eaa4e97af1282dbcf700410, user_id=716402a2fc9244b8847e832aea2deb77, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:23.022447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7688af9-baef-4398-8c21-dac850e31c49 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:23.037456 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1363/2764] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:22 2022] PUT /identity/v3/projects/0fe77dd73eaa4e97af1282dbcf700410/users/716402a2fc9244b8847e832aea2deb77/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:23.041534 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a74475-3b71-44c7-b3b3-f9bf440ab013 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:23.052298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a74475-3b71-44c7-b3b3-f9bf440ab013 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:23.112442 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ff7ed-0355-4ded-a0a0-77d04b880eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:23.113134 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ff7ed-0355-4ded-a0a0-77d04b880eaa admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:23.113398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ff7ed-0355-4ded-a0a0-77d04b880eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:23.113670 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ff7ed-0355-4ded-a0a0-77d04b880eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:23.114441 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ff7ed-0355-4ded-a0a0-77d04b880eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:23.116355 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ff7ed-0355-4ded-a0a0-77d04b880eaa admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:23.127077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1402/2765] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:23.131152 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7688af9-baef-4398-8c21-dac850e31c49 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:23.142230 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7688af9-baef-4398-8c21-dac850e31c49 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:23.148090 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a790d6-343f-4a07-98fe-09a259060e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:23.149087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a790d6-343f-4a07-98fe-09a259060e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:23.149441 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a790d6-343f-4a07-98fe-09a259060e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:23.149916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a790d6-343f-4a07-98fe-09a259060e68 admin admin] PATH_INFO: `/v3/projects/0fe77dd73eaa4e97af1282dbcf700410/users/716402a2fc9244b8847e832aea2deb77/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:23.152853 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a790d6-343f-4a07-98fe-09a259060e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe77dd73eaa4e97af1282dbcf700410, user_id=716402a2fc9244b8847e832aea2deb77, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:23.156274 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a790d6-343f-4a07-98fe-09a259060e68 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:23.168475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1364/2766] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:23 2022] PUT /identity/v3/projects/0fe77dd73eaa4e97af1282dbcf700410/users/716402a2fc9244b8847e832aea2deb77/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:23.179270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f2942-7cb8-41cf-891a-f600bc19a930 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:23.179557 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f2942-7cb8-41cf-891a-f600bc19a930 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:23.179858 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f2942-7cb8-41cf-891a-f600bc19a930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:23.207352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-b77f2942-7cb8-41cf-891a-f600bc19a930 None None] MFA Rules not processed for user `716402a2fc9244b8847e832aea2deb77`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:23.208617 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b77f2942-7cb8-41cf-891a-f600bc19a930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:23.285466 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1403/2767] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:23.296770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a790d6-343f-4a07-98fe-09a259060e68 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:23.298504 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-67a790d6-343f-4a07-98fe-09a259060e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:23.361633 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a790d6-343f-4a07-98fe-09a259060e68 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:23.365484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b2c10-24e9-4f24-b781-0b1d0c946133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:23.366148 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b2c10-24e9-4f24-b781-0b1d0c946133 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:23.366432 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b2c10-24e9-4f24-b781-0b1d0c946133 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:23.366709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b2c10-24e9-4f24-b781-0b1d0c946133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:23.425881 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b2c10-24e9-4f24-b781-0b1d0c946133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:23.427315 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b2c10-24e9-4f24-b781-0b1d0c946133 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:23.437919 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1365/2768] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:23.787623 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec295901-8032-4fd1-ae0b-0628c1bd47c8 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:23.787945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec295901-8032-4fd1-ae0b-0628c1bd47c8 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:23.788233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec295901-8032-4fd1-ae0b-0628c1bd47c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:23.814619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-ec295901-8032-4fd1-ae0b-0628c1bd47c8 None None] MFA Rules not processed for user `12b81c49a65e4059b52a4abe2966933a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:23.815870 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ec295901-8032-4fd1-ae0b-0628c1bd47c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:23.878756 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1404/2769] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:23.883234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b2c10-24e9-4f24-b781-0b1d0c946133 service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:23.947475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b2c10-24e9-4f24-b781-0b1d0c946133 service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:23.951886 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a736f80-2afb-4e87-ac80-c284bcbac979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:23.953475 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a736f80-2afb-4e87-ac80-c284bcbac979 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:23.953818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a736f80-2afb-4e87-ac80-c284bcbac979 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:23.954093 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a736f80-2afb-4e87-ac80-c284bcbac979 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:23.954778 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a736f80-2afb-4e87-ac80-c284bcbac979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:23.956699 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a736f80-2afb-4e87-ac80-c284bcbac979 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:23.981818 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1366/2770] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:34:23 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:23.985981 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec295901-8032-4fd1-ae0b-0628c1bd47c8 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:23.996287 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec295901-8032-4fd1-ae0b-0628c1bd47c8 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:24.050953 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b5fc7-8937-42c7-a433-1132da935c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:24.051615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b5fc7-8937-42c7-a433-1132da935c74 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:24.051923 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b5fc7-8937-42c7-a433-1132da935c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:24.052196 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b5fc7-8937-42c7-a433-1132da935c74 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:24.052874 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b5fc7-8937-42c7-a433-1132da935c74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:24.054760 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b5fc7-8937-42c7-a433-1132da935c74 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:24.073194 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1405/2771] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:34:23 2022] POST /identity/v3/users => generated 293 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:24.077773 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a736f80-2afb-4e87-ac80-c284bcbac979 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:24.087995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a736f80-2afb-4e87-ac80-c284bcbac979 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:24.093375 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a1935-546c-42b1-865a-85477ac7eefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:24.094132 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a1935-546c-42b1-865a-85477ac7eefe admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:24.094418 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a1935-546c-42b1-865a-85477ac7eefe admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:24.094696 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a1935-546c-42b1-865a-85477ac7eefe admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:24.095424 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a1935-546c-42b1-865a-85477ac7eefe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:24.097031 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a1935-546c-42b1-865a-85477ac7eefe admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:24.107159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1367/2772] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:24.111498 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b5fc7-8937-42c7-a433-1132da935c74 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:24.121016 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b5fc7-8937-42c7-a433-1132da935c74 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:24.125697 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541536c1-8819-496b-ba14-93c73a00715f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:24.126564 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541536c1-8819-496b-ba14-93c73a00715f admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:24.126842 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541536c1-8819-496b-ba14-93c73a00715f admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:24.127115 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541536c1-8819-496b-ba14-93c73a00715f admin admin] PATH_INFO: `/v3/projects/a7199d9ef46d47d584f6313945f60441/users/abe077a0928b4e02b6d8879e3c153c1d/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:24.141547 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541536c1-8819-496b-ba14-93c73a00715f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7199d9ef46d47d584f6313945f60441, user_id=abe077a0928b4e02b6d8879e3c153c1d, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:24.144189 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541536c1-8819-496b-ba14-93c73a00715f admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:24.157807 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1406/2773] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:24 2022] PUT /identity/v3/projects/a7199d9ef46d47d584f6313945f60441/users/abe077a0928b4e02b6d8879e3c153c1d/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:24.161873 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a1935-546c-42b1-865a-85477ac7eefe admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:24.173282 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a1935-546c-42b1-865a-85477ac7eefe admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:24.235755 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92a6d1-1266-49b6-807a-9de21707b250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:24.236469 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92a6d1-1266-49b6-807a-9de21707b250 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:24.236762 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92a6d1-1266-49b6-807a-9de21707b250 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:24.237034 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92a6d1-1266-49b6-807a-9de21707b250 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:24.237759 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92a6d1-1266-49b6-807a-9de21707b250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:24.239346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92a6d1-1266-49b6-807a-9de21707b250 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:24.254119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1368/2774] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:24.255317 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541536c1-8819-496b-ba14-93c73a00715f admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:24.267010 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541536c1-8819-496b-ba14-93c73a00715f admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:24.275245 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d095fe5-6f03-45bf-be23-84fd1b01ef85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:24.276583 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d095fe5-6f03-45bf-be23-84fd1b01ef85 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:24.276939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d095fe5-6f03-45bf-be23-84fd1b01ef85 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:24.277283 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d095fe5-6f03-45bf-be23-84fd1b01ef85 admin admin] PATH_INFO: `/v3/projects/a7199d9ef46d47d584f6313945f60441/users/abe077a0928b4e02b6d8879e3c153c1d/roles/e59113c0366049e889df3f55eed90c72` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:24.279385 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d095fe5-6f03-45bf-be23-84fd1b01ef85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7199d9ef46d47d584f6313945f60441, user_id=abe077a0928b4e02b6d8879e3c153c1d, role_id=e59113c0366049e889df3f55eed90c72)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:24.283708 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d095fe5-6f03-45bf-be23-84fd1b01ef85 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:24.294477 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1407/2775] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:24 2022] PUT /identity/v3/projects/a7199d9ef46d47d584f6313945f60441/users/abe077a0928b4e02b6d8879e3c153c1d/roles/e59113c0366049e889df3f55eed90c72 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:25.100681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369cd6d-5e65-45fe-812e-bc36c515ead9 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:25.101254 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369cd6d-5e65-45fe-812e-bc36c515ead9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:25.101543 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369cd6d-5e65-45fe-812e-bc36c515ead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:25.138503 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d369cd6d-5e65-45fe-812e-bc36c515ead9 None None] MFA Rules not processed for user `abe077a0928b4e02b6d8879e3c153c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:25.139843 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d369cd6d-5e65-45fe-812e-bc36c515ead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:25.209951 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1369/2776] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:25.222368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d095fe5-6f03-45bf-be23-84fd1b01ef85 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:25.223767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6d095fe5-6f03-45bf-be23-84fd1b01ef85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:25.282586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d095fe5-6f03-45bf-be23-84fd1b01ef85 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:25.287417 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c683e-ce26-4490-bf36-41ae2b989d5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:25.288172 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c683e-ce26-4490-bf36-41ae2b989d5c service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:25.288456 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c683e-ce26-4490-bf36-41ae2b989d5c service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:25.288728 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c683e-ce26-4490-bf36-41ae2b989d5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:25.344967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c683e-ce26-4490-bf36-41ae2b989d5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:25.346405 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c683e-ce26-4490-bf36-41ae2b989d5c service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:25.355027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1408/2777] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:26.010449 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369cd6d-5e65-45fe-812e-bc36c515ead9 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:26.071973 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369cd6d-5e65-45fe-812e-bc36c515ead9 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:26.076748 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:26.077956 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:26.078243 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:26.078523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] PATH_INFO: `/v3/users/309ce30960ff493795d913821c558c70` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:26.079727 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] RBAC: Authorizing `identity:delete_user(user_id=309ce30960ff493795d913821c558c70)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:26.081689 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:26.127550 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '309ce30960ff493795d913821c558c70'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.128385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '309ce30960ff493795d913821c558c70'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.128988 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '309ce30960ff493795d913821c558c70'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.140329 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '309ce30960ff493795d913821c558c70'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.147265 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '309ce30960ff493795d913821c558c70'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.167057 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1370/2778] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:34:26 2022] DELETE /identity/v3/users/309ce30960ff493795d913821c558c70 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:26.172338 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f9240-7e63-4094-9682-448adf61475d None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:26.172705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f9240-7e63-4094-9682-448adf61475d None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:26.173056 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f9240-7e63-4094-9682-448adf61475d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:26.189950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-de8f9240-7e63-4094-9682-448adf61475d None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:26.191241 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-de8f9240-7e63-4094-9682-448adf61475d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:26.251113 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1409/2779] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:26.260740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:26.262162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:26.318670 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e8f25-3e65-4c01-8487-9c75518fe11d admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:26.322221 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79267ed6-386d-477b-ba17-442808494603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:26.322857 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79267ed6-386d-477b-ba17-442808494603 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:26.323137 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79267ed6-386d-477b-ba17-442808494603 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:26.323410 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79267ed6-386d-477b-ba17-442808494603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:26.333327 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79267ed6-386d-477b-ba17-442808494603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:26.334725 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79267ed6-386d-477b-ba17-442808494603 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:26.344631 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1371/2780] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:26.602527 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f9240-7e63-4094-9682-448adf61475d None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:26.604109 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-de8f9240-7e63-4094-9682-448adf61475d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:26.619398 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f9240-7e63-4094-9682-448adf61475d None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:26.624855 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:26.625853 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:26.626135 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:26.626619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] PATH_INFO: `/v3/projects/682bf69acd354afd832f1b951ded2afe` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:26.628358 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=682bf69acd354afd832f1b951ded2afe)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:26.630444 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:26.679056 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '682bf69acd354afd832f1b951ded2afe', 'request_id': 'req-2602573e-7e53-4202-9774-a7db5b1e2afd'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.684264 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '682bf69acd354afd832f1b951ded2afe', 'request_id': 'req-2602573e-7e53-4202-9774-a7db5b1e2afd'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.685437 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79267ed6-386d-477b-ba17-442808494603 service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:26.687434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-79267ed6-386d-477b-ba17-442808494603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:26.692523 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '682bf69acd354afd832f1b951ded2afe', 'request_id': 'req-2602573e-7e53-4202-9774-a7db5b1e2afd'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.703310 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] The token cache is being invalidate because project 682bf69acd354afd832f1b951ded2afe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:34:26.703582 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.704298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.705869 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1410/2781] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:34:26 2022] DELETE /identity/v3/projects/682bf69acd354afd832f1b951ded2afe => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:34:26.709656 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:26.710956 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:26.745162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79267ed6-386d-477b-ba17-442808494603 service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:26.773939 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602573e-7e53-4202-9774-a7db5b1e2afd admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:26.786618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:26.787427 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:26.789987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:26.790271 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] PATH_INFO: `/v3/users/716402a2fc9244b8847e832aea2deb77` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:26.791662 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=716402a2fc9244b8847e832aea2deb77)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:26.793667 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:26.803602 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e4537-8f95-45ae-b336-852f9a3026e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:26.804299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e4537-8f95-45ae-b336-852f9a3026e0 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:26.804812 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e4537-8f95-45ae-b336-852f9a3026e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:26.805245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e4537-8f95-45ae-b336-852f9a3026e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:26.807264 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-723e4537-8f95-45ae-b336-852f9a3026e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:26.846495 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '716402a2fc9244b8847e832aea2deb77'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.847166 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '716402a2fc9244b8847e832aea2deb77'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.860608 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '716402a2fc9244b8847e832aea2deb77'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.868371 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '716402a2fc9244b8847e832aea2deb77'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.869101 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '716402a2fc9244b8847e832aea2deb77'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:26.884016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e4537-8f95-45ae-b336-852f9a3026e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:26.887128 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1411/2782] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:34:26 2022] DELETE /identity/v3/users/716402a2fc9244b8847e832aea2deb77 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:34:26.887279 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e4537-8f95-45ae-b336-852f9a3026e0 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:26.897865 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1372/2783] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:27.224369 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:27.227202 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:27.332711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5583a7c-2ca6-497f-9bba-1e0f5305d5aa admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:27.337724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60799716-f11c-429c-a03d-69328f601b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:27.338707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60799716-f11c-429c-a03d-69328f601b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:27.339229 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60799716-f11c-429c-a03d-69328f601b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:27.339997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60799716-f11c-429c-a03d-69328f601b32 admin admin] PATH_INFO: `/v3/projects/0fe77dd73eaa4e97af1282dbcf700410` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:27.343309 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60799716-f11c-429c-a03d-69328f601b32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe77dd73eaa4e97af1282dbcf700410)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:27.349281 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60799716-f11c-429c-a03d-69328f601b32 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:27.424080 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-60799716-f11c-429c-a03d-69328f601b32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe77dd73eaa4e97af1282dbcf700410', 'request_id': 'req-60799716-f11c-429c-a03d-69328f601b32'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:27.429719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-60799716-f11c-429c-a03d-69328f601b32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe77dd73eaa4e97af1282dbcf700410', 'request_id': 'req-60799716-f11c-429c-a03d-69328f601b32'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:27.439131 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-60799716-f11c-429c-a03d-69328f601b32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe77dd73eaa4e97af1282dbcf700410', 'request_id': 'req-60799716-f11c-429c-a03d-69328f601b32'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:27.450349 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-60799716-f11c-429c-a03d-69328f601b32 admin admin] The token cache is being invalidate because project 0fe77dd73eaa4e97af1282dbcf700410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:34:27.450716 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-60799716-f11c-429c-a03d-69328f601b32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:27.451433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-60799716-f11c-429c-a03d-69328f601b32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:27.453933 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1412/2784] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:34:27 2022] DELETE /identity/v3/projects/0fe77dd73eaa4e97af1282dbcf700410 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:27.458897 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51323d7-16ff-47d6-98b8-979897b6af62 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:27.459370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51323d7-16ff-47d6-98b8-979897b6af62 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:27.459490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51323d7-16ff-47d6-98b8-979897b6af62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:27.477507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-a51323d7-16ff-47d6-98b8-979897b6af62 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:27.478777 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a51323d7-16ff-47d6-98b8-979897b6af62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:27.539286 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1373/2785] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:34:27.548261 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92167474-c7b5-4bce-9ebc-eca8187f3071 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:27.548604 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92167474-c7b5-4bce-9ebc-eca8187f3071 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:27.548883 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92167474-c7b5-4bce-9ebc-eca8187f3071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:27.567335 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-92167474-c7b5-4bce-9ebc-eca8187f3071 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:27.568536 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-92167474-c7b5-4bce-9ebc-eca8187f3071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:27.576176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1413/2786] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:27.580893 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51323d7-16ff-47d6-98b8-979897b6af62 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:27.590472 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51323d7-16ff-47d6-98b8-979897b6af62 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:27.595029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905f305-ea14-4f6d-9d55-83565663311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:27.595671 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905f305-ea14-4f6d-9d55-83565663311b admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:27.595984 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905f305-ea14-4f6d-9d55-83565663311b admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:27.596261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905f305-ea14-4f6d-9d55-83565663311b admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:27.596976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905f305-ea14-4f6d-9d55-83565663311b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:34:27.597278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905f305-ea14-4f6d-9d55-83565663311b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:27.598862 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905f305-ea14-4f6d-9d55-83565663311b admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:27.610046 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1374/2787] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:34:27.614185 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92167474-c7b5-4bce-9ebc-eca8187f3071 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:27.623043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92167474-c7b5-4bce-9ebc-eca8187f3071 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:27.628145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d246e-561d-4d53-8e16-7e469a38ab10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:27.628795 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d246e-561d-4d53-8e16-7e469a38ab10 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:27.629071 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d246e-561d-4d53-8e16-7e469a38ab10 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:27.629349 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d246e-561d-4d53-8e16-7e469a38ab10 admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:27.630031 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d246e-561d-4d53-8e16-7e469a38ab10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:27.632030 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d246e-561d-4d53-8e16-7e469a38ab10 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:27.656421 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1414/2788] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:34:27 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:34:27.660669 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905f305-ea14-4f6d-9d55-83565663311b admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:27.670446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905f305-ea14-4f6d-9d55-83565663311b admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:27.725245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a00ee2-7424-4849-8d92-8c7496218605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:27.725908 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a00ee2-7424-4849-8d92-8c7496218605 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:27.726190 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a00ee2-7424-4849-8d92-8c7496218605 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:27.726477 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a00ee2-7424-4849-8d92-8c7496218605 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:27.727284 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a00ee2-7424-4849-8d92-8c7496218605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:27.729252 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a00ee2-7424-4849-8d92-8c7496218605 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:27.749821 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1375/2789] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:34:27 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:27.753852 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d246e-561d-4d53-8e16-7e469a38ab10 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:27.764407 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d246e-561d-4d53-8e16-7e469a38ab10 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:27.769767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531040eb-4815-4217-af4a-6b867ed7aed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:27.770433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531040eb-4815-4217-af4a-6b867ed7aed7 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:27.770715 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531040eb-4815-4217-af4a-6b867ed7aed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:27.771023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531040eb-4815-4217-af4a-6b867ed7aed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:27.771752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531040eb-4815-4217-af4a-6b867ed7aed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:27.773381 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531040eb-4815-4217-af4a-6b867ed7aed7 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:27.785035 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1415/2790] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:27.789216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a00ee2-7424-4849-8d92-8c7496218605 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:27.800237 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a00ee2-7424-4849-8d92-8c7496218605 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:27.804953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67041edb-9bda-48cf-8850-326a7b0226cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:27.805656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67041edb-9bda-48cf-8850-326a7b0226cc admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:27.805938 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67041edb-9bda-48cf-8850-326a7b0226cc admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:27.806216 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67041edb-9bda-48cf-8850-326a7b0226cc admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:27.806948 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67041edb-9bda-48cf-8850-326a7b0226cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:27.808855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67041edb-9bda-48cf-8850-326a7b0226cc admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:27.820375 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1376/2791] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:34:27.824487 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531040eb-4815-4217-af4a-6b867ed7aed7 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:27.835395 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531040eb-4815-4217-af4a-6b867ed7aed7 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:27.840927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b75c4-b06a-40df-ad4d-75079aedba36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:27.841814 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b75c4-b06a-40df-ad4d-75079aedba36 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:27.842095 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b75c4-b06a-40df-ad4d-75079aedba36 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:27.842366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b75c4-b06a-40df-ad4d-75079aedba36 admin admin] PATH_INFO: `/v3/projects/4f18c596d649479882634272916d2261/users/42966542e2cb452f99d03b894d64e2cb/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:27.857297 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b75c4-b06a-40df-ad4d-75079aedba36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f18c596d649479882634272916d2261, user_id=42966542e2cb452f99d03b894d64e2cb, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:27.860657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b75c4-b06a-40df-ad4d-75079aedba36 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:27.872997 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1416/2792] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:27 2022] PUT /identity/v3/projects/4f18c596d649479882634272916d2261/users/42966542e2cb452f99d03b894d64e2cb/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:34:27.883774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f6008-00c5-40bd-b631-3467eee7b409 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:27.884104 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f6008-00c5-40bd-b631-3467eee7b409 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:27.884382 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f6008-00c5-40bd-b631-3467eee7b409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:27.911582 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-a51f6008-00c5-40bd-b631-3467eee7b409 None None] MFA Rules not processed for user `42966542e2cb452f99d03b894d64e2cb`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:27.912970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-a51f6008-00c5-40bd-b631-3467eee7b409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:27.975779 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1377/2793] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:27.987383 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b75c4-b06a-40df-ad4d-75079aedba36 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:27.989117 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b04b75c4-b06a-40df-ad4d-75079aedba36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:28.042411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b75c4-b06a-40df-ad4d-75079aedba36 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:28.046643 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c91e19-d00f-402b-a3c0-865baf5811a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '295bb46627e043989e6dba2805607c99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:28.047431 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c91e19-d00f-402b-a3c0-865baf5811a9 service glance] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:28.047715 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c91e19-d00f-402b-a3c0-865baf5811a9 service glance] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:28.048015 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c91e19-d00f-402b-a3c0-865baf5811a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:28.056133 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c91e19-d00f-402b-a3c0-865baf5811a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:28.057540 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c91e19-d00f-402b-a3c0-865baf5811a9 service glance] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:28.066727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1417/2794] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:28.493528 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a3211-286b-49ee-ae45-ea6e56e46313 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:28.493987 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a3211-286b-49ee-ae45-ea6e56e46313 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:28.494101 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a3211-286b-49ee-ae45-ea6e56e46313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:28.526872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-525a3211-286b-49ee-ae45-ea6e56e46313 None None] MFA Rules not processed for user `09dd259fed464b8c98749dd7e369e871`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:28.526872 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-525a3211-286b-49ee-ae45-ea6e56e46313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:28.620026 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1378/2795] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:33.265856 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c91e19-d00f-402b-a3c0-865baf5811a9 service glance] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:33.267345 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-73c91e19-d00f-402b-a3c0-865baf5811a9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:33.340246 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c91e19-d00f-402b-a3c0-865baf5811a9 service glance] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:33.346485 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b926d6-b898-4d78-acc3-287eb4cadcc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd41e05b7ef954dd596c0134b65434dd6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:33.347161 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b926d6-b898-4d78-acc3-287eb4cadcc0 service placement] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:33.348092 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b926d6-b898-4d78-acc3-287eb4cadcc0 service placement] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:33.348389 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b926d6-b898-4d78-acc3-287eb4cadcc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:33.350454 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-00b926d6-b898-4d78-acc3-287eb4cadcc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:33.365409 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b926d6-b898-4d78-acc3-287eb4cadcc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:33.367003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b926d6-b898-4d78-acc3-287eb4cadcc0 service placement] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:33.378950 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1418/2796] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:34:44.370618 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765e600-65bb-4705-a821-df2a64f92e74 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:44.371095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765e600-65bb-4705-a821-df2a64f92e74 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:44.371255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765e600-65bb-4705-a821-df2a64f92e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:44.388545 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-6765e600-65bb-4705-a821-df2a64f92e74 None None] MFA Rules not processed for user `ec0360cd80a040c5ae2f3f7a6ba01f75`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:44.390703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-6765e600-65bb-4705-a821-df2a64f92e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:44.447161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1379/2797] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:45.803015 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b926d6-b898-4d78-acc3-287eb4cadcc0 service placement] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:45.805294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-00b926d6-b898-4d78-acc3-287eb4cadcc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:45.880407 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b926d6-b898-4d78-acc3-287eb4cadcc0 service placement] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:45.885182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:45.886058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:45.886339 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:45.886612 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] PATH_INFO: `/v3/users/12b81c49a65e4059b52a4abe2966933a` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:45.887800 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] RBAC: Authorizing `identity:delete_user(user_id=12b81c49a65e4059b52a4abe2966933a)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:45.889748 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:45.934778 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12b81c49a65e4059b52a4abe2966933a'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:45.935491 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12b81c49a65e4059b52a4abe2966933a'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:45.944187 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12b81c49a65e4059b52a4abe2966933a'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:45.950784 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12b81c49a65e4059b52a4abe2966933a'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:45.951411 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12b81c49a65e4059b52a4abe2966933a'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:45.968401 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1419/2798] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:34:45 2022] DELETE /identity/v3/users/12b81c49a65e4059b52a4abe2966933a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:46.142988 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765e600-65bb-4705-a821-df2a64f92e74 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:46.160957 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765e600-65bb-4705-a821-df2a64f92e74 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:46.220258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:46.221382 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:46.222107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:46.222107 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] PATH_INFO: `/v3/projects/20b20f5add5d405fb3c51185fab9c35b` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:46.223507 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] RBAC: Authorizing `identity:delete_project(project_id=20b20f5add5d405fb3c51185fab9c35b)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:46.225446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:46.323878 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20b20f5add5d405fb3c51185fab9c35b', 'request_id': 'req-53192226-b1df-41de-8756-24d10d32025e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:46.331442 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20b20f5add5d405fb3c51185fab9c35b', 'request_id': 'req-53192226-b1df-41de-8756-24d10d32025e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:46.340679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20b20f5add5d405fb3c51185fab9c35b', 'request_id': 'req-53192226-b1df-41de-8756-24d10d32025e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:46.355914 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] The token cache is being invalidate because project 20b20f5add5d405fb3c51185fab9c35b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:34:46.356344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:46.357055 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:46.359332 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1380/2799] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:34:46 2022] DELETE /identity/v3/projects/20b20f5add5d405fb3c51185fab9c35b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:46.363607 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:46.366320 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:46.403862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7491b1e-476c-4ca6-8faf-91c56307113b admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:46.466480 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837a296-82bf-4aec-820e-5c744d3938ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:46.467380 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837a296-82bf-4aec-820e-5c744d3938ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:46.467765 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837a296-82bf-4aec-820e-5c744d3938ec admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:46.468147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837a296-82bf-4aec-820e-5c744d3938ec admin admin] PATH_INFO: `/v3/users/abe077a0928b4e02b6d8879e3c153c1d` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:46.469402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837a296-82bf-4aec-820e-5c744d3938ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=abe077a0928b4e02b6d8879e3c153c1d)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:46.471423 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837a296-82bf-4aec-820e-5c744d3938ec admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:46.518476 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d837a296-82bf-4aec-820e-5c744d3938ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abe077a0928b4e02b6d8879e3c153c1d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:46.519139 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d837a296-82bf-4aec-820e-5c744d3938ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abe077a0928b4e02b6d8879e3c153c1d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:46.526450 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d837a296-82bf-4aec-820e-5c744d3938ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abe077a0928b4e02b6d8879e3c153c1d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:46.533513 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d837a296-82bf-4aec-820e-5c744d3938ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abe077a0928b4e02b6d8879e3c153c1d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:46.534169 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-d837a296-82bf-4aec-820e-5c744d3938ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abe077a0928b4e02b6d8879e3c153c1d'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:46.563967 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1420/2800] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:34:46 2022] DELETE /identity/v3/users/abe077a0928b4e02b6d8879e3c153c1d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:46.887978 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:46.889557 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:46.903837 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53192226-b1df-41de-8756-24d10d32025e admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:46.966297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c798cf7-a890-40df-afd7-cc51617d1981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:46.967338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c798cf7-a890-40df-afd7-cc51617d1981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:46.968180 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c798cf7-a890-40df-afd7-cc51617d1981 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:46.968491 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c798cf7-a890-40df-afd7-cc51617d1981 admin admin] PATH_INFO: `/v3/projects/a7199d9ef46d47d584f6313945f60441` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:46.969740 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c798cf7-a890-40df-afd7-cc51617d1981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7199d9ef46d47d584f6313945f60441)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:46.971656 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c798cf7-a890-40df-afd7-cc51617d1981 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:47.017434 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1c798cf7-a890-40df-afd7-cc51617d1981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7199d9ef46d47d584f6313945f60441', 'request_id': 'req-1c798cf7-a890-40df-afd7-cc51617d1981'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:47.023682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1c798cf7-a890-40df-afd7-cc51617d1981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7199d9ef46d47d584f6313945f60441', 'request_id': 'req-1c798cf7-a890-40df-afd7-cc51617d1981'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:47.029261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1c798cf7-a890-40df-afd7-cc51617d1981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7199d9ef46d47d584f6313945f60441', 'request_id': 'req-1c798cf7-a890-40df-afd7-cc51617d1981'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:47.042666 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1c798cf7-a890-40df-afd7-cc51617d1981 admin admin] The token cache is being invalidate because project a7199d9ef46d47d584f6313945f60441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:34:47.042953 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1c798cf7-a890-40df-afd7-cc51617d1981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:47.044059 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-1c798cf7-a890-40df-afd7-cc51617d1981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:34:47.046845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1381/2801] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:34:46 2022] DELETE /identity/v3/projects/a7199d9ef46d47d584f6313945f60441 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:47.051611 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd1c09-843c-48df-b5ca-c3920b6ccb0a None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:47.052199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd1c09-843c-48df-b5ca-c3920b6ccb0a None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:47.052312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd1c09-843c-48df-b5ca-c3920b6ccb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:47.071539 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-cfdd1c09-843c-48df-b5ca-c3920b6ccb0a None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:47.072787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-cfdd1c09-843c-48df-b5ca-c3920b6ccb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:47.132343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1421/2802] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:34:47.194651 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc6fb6-889b-47c3-8087-49605c62b93e None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:47.195121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc6fb6-889b-47c3-8087-49605c62b93e None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:47.195217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc6fb6-889b-47c3-8087-49605c62b93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:47.213802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-8ebc6fb6-889b-47c3-8087-49605c62b93e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:47.215261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8ebc6fb6-889b-47c3-8087-49605c62b93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:47.223657 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1382/2803] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:47.228298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd1c09-843c-48df-b5ca-c3920b6ccb0a None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:47.238464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd1c09-843c-48df-b5ca-c3920b6ccb0a None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:47.243059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57809ba4-7cbc-4302-8502-2edc06562791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:47.243707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57809ba4-7cbc-4302-8502-2edc06562791 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:47.244023 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57809ba4-7cbc-4302-8502-2edc06562791 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:47.244298 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57809ba4-7cbc-4302-8502-2edc06562791 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:47.245019 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57809ba4-7cbc-4302-8502-2edc06562791 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:34:47.245322 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57809ba4-7cbc-4302-8502-2edc06562791 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:47.246903 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57809ba4-7cbc-4302-8502-2edc06562791 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:47.258934 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1422/2804] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:34:47.263446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc6fb6-889b-47c3-8087-49605c62b93e None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:47.273346 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc6fb6-889b-47c3-8087-49605c62b93e None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:47.277926 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d4dd3-4d64-45ab-8c00-fda94b2bbbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:47.278575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d4dd3-4d64-45ab-8c00-fda94b2bbbf9 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:47.278860 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d4dd3-4d64-45ab-8c00-fda94b2bbbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:47.279136 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d4dd3-4d64-45ab-8c00-fda94b2bbbf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:47.279828 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d4dd3-4d64-45ab-8c00-fda94b2bbbf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:47.282496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d4dd3-4d64-45ab-8c00-fda94b2bbbf9 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:47.307601 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1383/2805] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:34:47 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:47.311661 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57809ba4-7cbc-4302-8502-2edc06562791 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:47.321461 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57809ba4-7cbc-4302-8502-2edc06562791 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:47.376059 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed560d-07c1-488b-b32f-a25bcda9f771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:47.376719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed560d-07c1-488b-b32f-a25bcda9f771 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:47.377003 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed560d-07c1-488b-b32f-a25bcda9f771 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:47.377279 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed560d-07c1-488b-b32f-a25bcda9f771 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:47.377960 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed560d-07c1-488b-b32f-a25bcda9f771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:47.379912 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed560d-07c1-488b-b32f-a25bcda9f771 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:47.397667 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1423/2806] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:34:47 2022] POST /identity/v3/users => generated 296 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:34:47.401786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d4dd3-4d64-45ab-8c00-fda94b2bbbf9 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:47.412574 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d4dd3-4d64-45ab-8c00-fda94b2bbbf9 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:47.417343 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30143b3-3519-46bd-8362-f3fb97368dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:47.418011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30143b3-3519-46bd-8362-f3fb97368dbd admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:47.418369 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30143b3-3519-46bd-8362-f3fb97368dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:47.418555 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30143b3-3519-46bd-8362-f3fb97368dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:47.419278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30143b3-3519-46bd-8362-f3fb97368dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:47.421409 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30143b3-3519-46bd-8362-f3fb97368dbd admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:47.433047 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1384/2807] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:34:47.438061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed560d-07c1-488b-b32f-a25bcda9f771 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:47.447779 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed560d-07c1-488b-b32f-a25bcda9f771 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:47.452602 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c630067-205d-402d-b48a-9c3eec6943a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:47.453384 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c630067-205d-402d-b48a-9c3eec6943a3 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:47.453659 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c630067-205d-402d-b48a-9c3eec6943a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:47.453964 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c630067-205d-402d-b48a-9c3eec6943a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:47.454703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c630067-205d-402d-b48a-9c3eec6943a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:47.456324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c630067-205d-402d-b48a-9c3eec6943a3 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:47.466652 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1424/2808] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 04:34:47.470887 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30143b3-3519-46bd-8362-f3fb97368dbd admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:47.480952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30143b3-3519-46bd-8362-f3fb97368dbd admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:47.487515 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98108bc8-9d81-47d6-858c-3237153a50a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:47.488451 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98108bc8-9d81-47d6-858c-3237153a50a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:47.489005 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98108bc8-9d81-47d6-858c-3237153a50a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:47.489005 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98108bc8-9d81-47d6-858c-3237153a50a0 admin admin] PATH_INFO: `/v3/projects/313a26894ef3432f8d1bf1866a1c1bf3/users/ea22fbf03b3f4a78bd9d0e6d3de35967/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:47.507258 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98108bc8-9d81-47d6-858c-3237153a50a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=313a26894ef3432f8d1bf1866a1c1bf3, user_id=ea22fbf03b3f4a78bd9d0e6d3de35967, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:47.509924 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98108bc8-9d81-47d6-858c-3237153a50a0 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:47.527431 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1385/2809] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:34:47 2022] PUT /identity/v3/projects/313a26894ef3432f8d1bf1866a1c1bf3/users/ea22fbf03b3f4a78bd9d0e6d3de35967/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:34:47.537483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407380a2-51b1-4a6b-b1f8-b1f7ba855730 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:47.537825 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407380a2-51b1-4a6b-b1f8-b1f7ba855730 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:47.538075 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407380a2-51b1-4a6b-b1f8-b1f7ba855730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:47.565776 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-407380a2-51b1-4a6b-b1f8-b1f7ba855730 None None] MFA Rules not processed for user `ea22fbf03b3f4a78bd9d0e6d3de35967`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:34:47.567145 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-407380a2-51b1-4a6b-b1f8-b1f7ba855730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:47.629985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1425/2810] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:34:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:34:47.640970 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98108bc8-9d81-47d6-858c-3237153a50a0 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:34:47.642751 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-98108bc8-9d81-47d6-858c-3237153a50a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:34:47.703138 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98108bc8-9d81-47d6-858c-3237153a50a0 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:34:47.707280 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa51bd0-62ed-4b42-8789-918aeab71a0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:34:47.707977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa51bd0-62ed-4b42-8789-918aeab71a0d service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:34:47.708241 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa51bd0-62ed-4b42-8789-918aeab71a0d service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:34:47.708520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa51bd0-62ed-4b42-8789-918aeab71a0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:34:47.720117 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa51bd0-62ed-4b42-8789-918aeab71a0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:34:47.721826 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa51bd0-62ed-4b42-8789-918aeab71a0d service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:34:47.731575 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1386/2811] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:34:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:35:07.034601 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439a0a-43e0-4168-bceb-ed5e4fef1164 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:07.035142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439a0a-43e0-4168-bceb-ed5e4fef1164 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:07.035142 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439a0a-43e0-4168-bceb-ed5e4fef1164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:07.052370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2f439a0a-43e0-4168-bceb-ed5e4fef1164 None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:07.053685 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2f439a0a-43e0-4168-bceb-ed5e4fef1164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:07.107807 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1426/2812] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:35:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:07.155233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa51bd0-62ed-4b42-8789-918aeab71a0d service cinder] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:07.156651 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-6fa51bd0-62ed-4b42-8789-918aeab71a0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:07.223142 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa51bd0-62ed-4b42-8789-918aeab71a0d service cinder] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:07.228048 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d146b6d-ad81-4417-91b8-979009ba140f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:07.228733 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d146b6d-ad81-4417-91b8-979009ba140f service swift] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:07.229018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d146b6d-ad81-4417-91b8-979009ba140f service swift] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:07.229293 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d146b6d-ad81-4417-91b8-979009ba140f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:07.238510 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d146b6d-ad81-4417-91b8-979009ba140f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:07.240106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d146b6d-ad81-4417-91b8-979009ba140f service swift] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:07.247995 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1387/2813] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:35:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:35:08.557844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f439a0a-43e0-4168-bceb-ed5e4fef1164 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:08.560205 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2f439a0a-43e0-4168-bceb-ed5e4fef1164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:08.637100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f439a0a-43e0-4168-bceb-ed5e4fef1164 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:08.641723 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:08.642831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:08.643113 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:08.643546 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] PATH_INFO: `/v3/users/42966542e2cb452f99d03b894d64e2cb` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:08.644944 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42966542e2cb452f99d03b894d64e2cb)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:08.646850 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:08.699464 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42966542e2cb452f99d03b894d64e2cb'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:08.700267 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42966542e2cb452f99d03b894d64e2cb'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:08.709433 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42966542e2cb452f99d03b894d64e2cb'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:08.722174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42966542e2cb452f99d03b894d64e2cb'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:08.723111 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42966542e2cb452f99d03b894d64e2cb'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:08.748324 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1427/2814] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:35:08 2022] DELETE /identity/v3/users/42966542e2cb452f99d03b894d64e2cb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:35:08.754830 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a73f70-8b1e-4af1-9579-1962a2679e08 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:08.755978 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a73f70-8b1e-4af1-9579-1962a2679e08 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:08.756681 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a73f70-8b1e-4af1-9579-1962a2679e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:08.781559 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-33a73f70-8b1e-4af1-9579-1962a2679e08 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:08.783367 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-33a73f70-8b1e-4af1-9579-1962a2679e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:08.864075 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1388/2815] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:08.872560 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:08.874866 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:08.953186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50edd6-07b8-4afd-a0d4-e6037107f0c0 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:08.957646 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a0f37-0ae3-4038-ab38-034e158dbcae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:08.958269 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a0f37-0ae3-4038-ab38-034e158dbcae service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:08.958928 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a0f37-0ae3-4038-ab38-034e158dbcae service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:08.959294 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a0f37-0ae3-4038-ab38-034e158dbcae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:08.970208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a0f37-0ae3-4038-ab38-034e158dbcae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:08.972704 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a0f37-0ae3-4038-ab38-034e158dbcae service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:08.982879 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1428/2816] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:35:09.347774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a73f70-8b1e-4af1-9579-1962a2679e08 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:09.349821 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-33a73f70-8b1e-4af1-9579-1962a2679e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:09.372712 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a73f70-8b1e-4af1-9579-1962a2679e08 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:09.380624 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231313d-45a7-4ed1-8e68-3557f8a74562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:09.382198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231313d-45a7-4ed1-8e68-3557f8a74562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:09.382682 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231313d-45a7-4ed1-8e68-3557f8a74562 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:09.383088 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231313d-45a7-4ed1-8e68-3557f8a74562 admin admin] PATH_INFO: `/v3/projects/4f18c596d649479882634272916d2261` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:09.395494 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231313d-45a7-4ed1-8e68-3557f8a74562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f18c596d649479882634272916d2261)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:09.401008 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231313d-45a7-4ed1-8e68-3557f8a74562 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:09.489311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e231313d-45a7-4ed1-8e68-3557f8a74562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f18c596d649479882634272916d2261', 'request_id': 'req-e231313d-45a7-4ed1-8e68-3557f8a74562'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:09.496349 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e231313d-45a7-4ed1-8e68-3557f8a74562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f18c596d649479882634272916d2261', 'request_id': 'req-e231313d-45a7-4ed1-8e68-3557f8a74562'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:09.504770 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e231313d-45a7-4ed1-8e68-3557f8a74562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f18c596d649479882634272916d2261', 'request_id': 'req-e231313d-45a7-4ed1-8e68-3557f8a74562'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:09.522614 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e231313d-45a7-4ed1-8e68-3557f8a74562 admin admin] The token cache is being invalidate because project 4f18c596d649479882634272916d2261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:35:09.522957 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e231313d-45a7-4ed1-8e68-3557f8a74562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:09.523665 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e231313d-45a7-4ed1-8e68-3557f8a74562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:09.526310 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1389/2817] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:35:09 2022] DELETE /identity/v3/projects/4f18c596d649479882634272916d2261 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:35:09.531346 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f9a81-6028-4c3a-b457-f5fd50098741 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:09.531765 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f9a81-6028-4c3a-b457-f5fd50098741 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:09.532233 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f9a81-6028-4c3a-b457-f5fd50098741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:09.552768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-6c9f9a81-6028-4c3a-b457-f5fd50098741 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:09.554155 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-6c9f9a81-6028-4c3a-b457-f5fd50098741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:09.619941 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1429/2818] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:09.625176 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ed74e-aba2-4e72-b961-5dd30dbfaa84 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:09.625493 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ed74e-aba2-4e72-b961-5dd30dbfaa84 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:09.625786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ed74e-aba2-4e72-b961-5dd30dbfaa84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:09.645786 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-455ed74e-aba2-4e72-b961-5dd30dbfaa84 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:09.646980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-455ed74e-aba2-4e72-b961-5dd30dbfaa84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:09.655188 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1390/2819] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:09.659578 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f9a81-6028-4c3a-b457-f5fd50098741 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:09.669927 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f9a81-6028-4c3a-b457-f5fd50098741 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:09.675857 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d1439-d356-40f6-8a0a-041e31796de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:09.676589 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d1439-d356-40f6-8a0a-041e31796de0 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:09.676843 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d1439-d356-40f6-8a0a-041e31796de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:09.677170 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d1439-d356-40f6-8a0a-041e31796de0 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:09.678001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d1439-d356-40f6-8a0a-041e31796de0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:35:09.678314 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d1439-d356-40f6-8a0a-041e31796de0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:09.682058 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d1439-d356-40f6-8a0a-041e31796de0 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:09.695037 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1430/2820] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:35:09.699305 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ed74e-aba2-4e72-b961-5dd30dbfaa84 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:09.711311 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ed74e-aba2-4e72-b961-5dd30dbfaa84 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:09.718560 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcd2cb-6576-42c2-a5a1-04edcb36a4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:09.719399 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcd2cb-6576-42c2-a5a1-04edcb36a4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:09.719845 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcd2cb-6576-42c2-a5a1-04edcb36a4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:09.720342 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcd2cb-6576-42c2-a5a1-04edcb36a4f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:09.721035 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fcd2cb-6576-42c2-a5a1-04edcb36a4f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:09.724367 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fcd2cb-6576-42c2-a5a1-04edcb36a4f6 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:09.766256 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1391/2821] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:35:09 2022] POST /identity/v3/projects => generated 363 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:35:09.771202 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d1439-d356-40f6-8a0a-041e31796de0 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:09.783542 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d1439-d356-40f6-8a0a-041e31796de0 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:09.859727 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6ee01-bb69-4623-855a-536d5f130063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:09.860437 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6ee01-bb69-4623-855a-536d5f130063 admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:09.860722 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6ee01-bb69-4623-855a-536d5f130063 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:09.861000 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6ee01-bb69-4623-855a-536d5f130063 admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:09.861705 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6ee01-bb69-4623-855a-536d5f130063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:09.864458 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6ee01-bb69-4623-855a-536d5f130063 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:09.887935 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1431/2822] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:35:09 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 04:35:09.891159 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcd2cb-6576-42c2-a5a1-04edcb36a4f6 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:09.903650 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcd2cb-6576-42c2-a5a1-04edcb36a4f6 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:09.910149 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c976183-32f5-4e68-9fb8-58bacdc0ddc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:09.911230 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c976183-32f5-4e68-9fb8-58bacdc0ddc1 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:09.911849 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c976183-32f5-4e68-9fb8-58bacdc0ddc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:09.912613 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c976183-32f5-4e68-9fb8-58bacdc0ddc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:09.913804 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c976183-32f5-4e68-9fb8-58bacdc0ddc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:09.916017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c976183-32f5-4e68-9fb8-58bacdc0ddc1 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:09.930016 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1392/2823] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:35:09.933639 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6ee01-bb69-4623-855a-536d5f130063 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:09.946149 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6ee01-bb69-4623-855a-536d5f130063 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:09.953360 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4d224-d42a-414c-b165-78e068145447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:09.954168 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4d224-d42a-414c-b165-78e068145447 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:09.954622 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4d224-d42a-414c-b165-78e068145447 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:09.954963 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4d224-d42a-414c-b165-78e068145447 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:09.955802 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4d224-d42a-414c-b165-78e068145447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:09.958009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4d224-d42a-414c-b165-78e068145447 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:09.982128 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1432/2824] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:35:09.985745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c976183-32f5-4e68-9fb8-58bacdc0ddc1 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:10.005235 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c976183-32f5-4e68-9fb8-58bacdc0ddc1 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:10.012297 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07721dd-653d-4be8-a09d-98126bb74f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:10.013463 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07721dd-653d-4be8-a09d-98126bb74f28 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:10.013754 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07721dd-653d-4be8-a09d-98126bb74f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:10.014029 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07721dd-653d-4be8-a09d-98126bb74f28 admin admin] PATH_INFO: `/v3/projects/62bb515e67e64fa99ed96a95b5aee9b8/users/56f5b7db02674c6f80ef61d2410e9208/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:10.033855 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07721dd-653d-4be8-a09d-98126bb74f28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62bb515e67e64fa99ed96a95b5aee9b8, user_id=56f5b7db02674c6f80ef61d2410e9208, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:10.037068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07721dd-653d-4be8-a09d-98126bb74f28 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:10.051068 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1393/2825] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:35:09 2022] PUT /identity/v3/projects/62bb515e67e64fa99ed96a95b5aee9b8/users/56f5b7db02674c6f80ef61d2410e9208/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:35:10.061671 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5954805-709d-4533-91c0-0378f9eb9ca5 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:10.061958 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5954805-709d-4533-91c0-0378f9eb9ca5 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:10.062234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5954805-709d-4533-91c0-0378f9eb9ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:10.093042 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-b5954805-709d-4533-91c0-0378f9eb9ca5 None None] MFA Rules not processed for user `56f5b7db02674c6f80ef61d2410e9208`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:10.094285 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b5954805-709d-4533-91c0-0378f9eb9ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:10.188151 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1433/2826] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:10 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:10.199980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07721dd-653d-4be8-a09d-98126bb74f28 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:10.201794 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-f07721dd-653d-4be8-a09d-98126bb74f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:10.278683 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07721dd-653d-4be8-a09d-98126bb74f28 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:10.282446 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc07d5-7fa1-4e2a-a504-3402a6bfe2e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:10.283106 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc07d5-7fa1-4e2a-a504-3402a6bfe2e7 service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:10.283393 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc07d5-7fa1-4e2a-a504-3402a6bfe2e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:10.283672 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc07d5-7fa1-4e2a-a504-3402a6bfe2e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:10.293049 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc07d5-7fa1-4e2a-a504-3402a6bfe2e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:10.294554 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc07d5-7fa1-4e2a-a504-3402a6bfe2e7 service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:10.305151 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1394/2827] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:35:10 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:35:11.318295 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5954805-709d-4533-91c0-0378f9eb9ca5 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:11.320678 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-b5954805-709d-4533-91c0-0378f9eb9ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:11.406793 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5954805-709d-4533-91c0-0378f9eb9ca5 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:11.415262 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:11.416128 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:11.416420 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:11.416668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] PATH_INFO: `/v3/users/ea22fbf03b3f4a78bd9d0e6d3de35967` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:11.417846 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea22fbf03b3f4a78bd9d0e6d3de35967)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:11.419774 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:11.470354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea22fbf03b3f4a78bd9d0e6d3de35967'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:11.471215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea22fbf03b3f4a78bd9d0e6d3de35967'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:11.480235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea22fbf03b3f4a78bd9d0e6d3de35967'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:11.488486 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea22fbf03b3f4a78bd9d0e6d3de35967'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:11.489244 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea22fbf03b3f4a78bd9d0e6d3de35967'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:11.507783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1434/2828] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:35:11 2022] DELETE /identity/v3/users/ea22fbf03b3f4a78bd9d0e6d3de35967 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:35:11.513228 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad6af5-a2b0-4453-b30d-f154825120b9 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:11.514223 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad6af5-a2b0-4453-b30d-f154825120b9 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:11.514499 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad6af5-a2b0-4453-b30d-f154825120b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:11.533687 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-b0ad6af5-a2b0-4453-b30d-f154825120b9 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:11.535468 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b0ad6af5-a2b0-4453-b30d-f154825120b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:11.610464 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1395/2829] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:11.622768 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:11.625034 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:11.698402 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46587631-2fa5-45d6-a1c5-4cfab449c9cb admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:11.702919 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f8254-0bf2-429c-92e2-4a1e1b64def2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:11.703561 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f8254-0bf2-429c-92e2-4a1e1b64def2 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:11.703844 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f8254-0bf2-429c-92e2-4a1e1b64def2 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:11.704148 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f8254-0bf2-429c-92e2-4a1e1b64def2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:11.714744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f8254-0bf2-429c-92e2-4a1e1b64def2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:11.716636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f8254-0bf2-429c-92e2-4a1e1b64def2 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:11.727816 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1435/2830] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:35:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:35:12.175869 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad6af5-a2b0-4453-b30d-f154825120b9 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:12.177819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b0ad6af5-a2b0-4453-b30d-f154825120b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:12.197140 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad6af5-a2b0-4453-b30d-f154825120b9 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:12.209198 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:12.210329 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:12.210795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:12.211387 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] PATH_INFO: `/v3/projects/313a26894ef3432f8d1bf1866a1c1bf3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:12.213119 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=313a26894ef3432f8d1bf1866a1c1bf3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:12.215584 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:12.279130 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '313a26894ef3432f8d1bf1866a1c1bf3', 'request_id': 'req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:12.290924 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '313a26894ef3432f8d1bf1866a1c1bf3', 'request_id': 'req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:12.296810 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '313a26894ef3432f8d1bf1866a1c1bf3', 'request_id': 'req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:12.316581 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] The token cache is being invalidate because project 313a26894ef3432f8d1bf1866a1c1bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:35:12.316929 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:12.317686 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:12.319993 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1396/2831] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:35:12 2022] DELETE /identity/v3/projects/313a26894ef3432f8d1bf1866a1c1bf3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:35:22.948613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f1b30-61ac-40ff-92ef-0c9daaf4a8b9 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:22.949651 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f1b30-61ac-40ff-92ef-0c9daaf4a8b9 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:22.950220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f1b30-61ac-40ff-92ef-0c9daaf4a8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:22.976062 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-1b4f1b30-61ac-40ff-92ef-0c9daaf4a8b9 None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:22.977299 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1b4f1b30-61ac-40ff-92ef-0c9daaf4a8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:23.042342 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1436/2832] 10.222.0.44 () {58 vars in 967 bytes} [Mon May 9 04:35:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:35:23.055294 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:23.056791 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:23.126543 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca1e89-38a6-4559-9106-bb8d3c580b1b admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:23.132144 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bc18c-9f70-43e6-ab43-3e0d052a3905 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:23.132825 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bc18c-9f70-43e6-ab43-3e0d052a3905 service swift] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:23.133110 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bc18c-9f70-43e6-ab43-3e0d052a3905 service swift] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:23.133389 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bc18c-9f70-43e6-ab43-3e0d052a3905 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:23.141358 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bc18c-9f70-43e6-ab43-3e0d052a3905 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:23.143430 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bc18c-9f70-43e6-ab43-3e0d052a3905 service swift] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:23.151865 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1397/2833] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:35:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:35:27.179077 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f1b30-61ac-40ff-92ef-0c9daaf4a8b9 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:27.180639 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-1b4f1b30-61ac-40ff-92ef-0c9daaf4a8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:27.245658 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f1b30-61ac-40ff-92ef-0c9daaf4a8b9 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:27.250242 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4a2b6-ece7-47e3-9760-70b5842c4a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:27.251108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4a2b6-ece7-47e3-9760-70b5842c4a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:27.251386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4a2b6-ece7-47e3-9760-70b5842c4a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:27.251654 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4a2b6-ece7-47e3-9760-70b5842c4a56 admin admin] PATH_INFO: `/v3/users/09dd259fed464b8c98749dd7e369e871` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:27.253507 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4a2b6-ece7-47e3-9760-70b5842c4a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09dd259fed464b8c98749dd7e369e871)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:27.255425 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4a2b6-ece7-47e3-9760-70b5842c4a56 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:27.298545 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-15f4a2b6-ece7-47e3-9760-70b5842c4a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09dd259fed464b8c98749dd7e369e871'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:27.299268 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-15f4a2b6-ece7-47e3-9760-70b5842c4a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09dd259fed464b8c98749dd7e369e871'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:27.306462 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-15f4a2b6-ece7-47e3-9760-70b5842c4a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09dd259fed464b8c98749dd7e369e871'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:27.313047 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-15f4a2b6-ece7-47e3-9760-70b5842c4a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09dd259fed464b8c98749dd7e369e871'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:27.313675 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-15f4a2b6-ece7-47e3-9760-70b5842c4a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09dd259fed464b8c98749dd7e369e871'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:27.330028 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1437/2834] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:35:27 2022] DELETE /identity/v3/users/09dd259fed464b8c98749dd7e369e871 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:35:27.501667 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bc18c-9f70-43e6-ab43-3e0d052a3905 service swift] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:27.512532 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bc18c-9f70-43e6-ab43-3e0d052a3905 service swift] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:27.567883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623d666-26eb-484b-9eaf-9a7107a03783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:27.568760 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623d666-26eb-484b-9eaf-9a7107a03783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:27.569036 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623d666-26eb-484b-9eaf-9a7107a03783 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:27.569302 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623d666-26eb-484b-9eaf-9a7107a03783 admin admin] PATH_INFO: `/v3/projects/20b03a05541e4ee38dc5f94a6e4bdc69` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:27.570447 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623d666-26eb-484b-9eaf-9a7107a03783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20b03a05541e4ee38dc5f94a6e4bdc69)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:27.572382 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623d666-26eb-484b-9eaf-9a7107a03783 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:27.616653 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4623d666-26eb-484b-9eaf-9a7107a03783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20b03a05541e4ee38dc5f94a6e4bdc69', 'request_id': 'req-4623d666-26eb-484b-9eaf-9a7107a03783'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:27.622550 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4623d666-26eb-484b-9eaf-9a7107a03783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20b03a05541e4ee38dc5f94a6e4bdc69', 'request_id': 'req-4623d666-26eb-484b-9eaf-9a7107a03783'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:27.628395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4623d666-26eb-484b-9eaf-9a7107a03783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20b03a05541e4ee38dc5f94a6e4bdc69', 'request_id': 'req-4623d666-26eb-484b-9eaf-9a7107a03783'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:27.639745 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4623d666-26eb-484b-9eaf-9a7107a03783 admin admin] The token cache is being invalidate because project 20b03a05541e4ee38dc5f94a6e4bdc69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:35:27.640075 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4623d666-26eb-484b-9eaf-9a7107a03783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:27.640690 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-4623d666-26eb-484b-9eaf-9a7107a03783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:27.642210 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1398/2835] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:35:27 2022] DELETE /identity/v3/projects/20b03a05541e4ee38dc5f94a6e4bdc69 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:35:27.647202 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16be56-848d-421b-b851-f95ac795deea None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:27.647493 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16be56-848d-421b-b851-f95ac795deea None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:27.647773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16be56-848d-421b-b851-f95ac795deea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:27.664849 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-ba16be56-848d-421b-b851-f95ac795deea None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:27.666041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-ba16be56-848d-421b-b851-f95ac795deea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:27.727292 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1438/2836] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:27.735539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca1f79-31ed-492f-914d-440259f52314 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:27.735984 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca1f79-31ed-492f-914d-440259f52314 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:27.736329 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca1f79-31ed-492f-914d-440259f52314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:27.763613 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-faca1f79-31ed-492f-914d-440259f52314 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:27.765017 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-faca1f79-31ed-492f-914d-440259f52314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:27.783233 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1399/2837] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 04:35:27.795043 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16be56-848d-421b-b851-f95ac795deea None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:27.806370 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16be56-848d-421b-b851-f95ac795deea None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:27.811744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82627f-8fad-486b-a572-1e1f24a1ce63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:27.812446 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82627f-8fad-486b-a572-1e1f24a1ce63 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:27.812785 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82627f-8fad-486b-a572-1e1f24a1ce63 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:27.813098 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82627f-8fad-486b-a572-1e1f24a1ce63 admin admin] PATH_INFO: `/v3/domains` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:27.814167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82627f-8fad-486b-a572-1e1f24a1ce63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79764) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:35:27.814167 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82627f-8fad-486b-a572-1e1f24a1ce63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:27.816354 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82627f-8fad-486b-a572-1e1f24a1ce63 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:27.829100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1439/2838] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 04:35:27.833910 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca1f79-31ed-492f-914d-440259f52314 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:27.848080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca1f79-31ed-492f-914d-440259f52314 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:27.853947 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920d2ac-e6a7-4d88-9e6f-8589daf5a2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:27.854743 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920d2ac-e6a7-4d88-9e6f-8589daf5a2db admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:27.855053 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920d2ac-e6a7-4d88-9e6f-8589daf5a2db admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:27.855377 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920d2ac-e6a7-4d88-9e6f-8589daf5a2db admin admin] PATH_INFO: `/v3/projects` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:27.856424 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920d2ac-e6a7-4d88-9e6f-8589daf5a2db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:27.858546 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920d2ac-e6a7-4d88-9e6f-8589daf5a2db admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:27.886072 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1400/2839] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:35:27 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:35:27.890235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82627f-8fad-486b-a572-1e1f24a1ce63 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:27.901650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82627f-8fad-486b-a572-1e1f24a1ce63 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:27.981874 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85720f-159e-45de-afcf-205686e51e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:27.982553 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85720f-159e-45de-afcf-205686e51e6e admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:27.982834 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85720f-159e-45de-afcf-205686e51e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:27.983140 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85720f-159e-45de-afcf-205686e51e6e admin admin] PATH_INFO: `/v3/users` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:27.983853 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85720f-159e-45de-afcf-205686e51e6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:27.985921 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85720f-159e-45de-afcf-205686e51e6e admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:28.010005 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1440/2840] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:35:27 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:35:28.014194 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920d2ac-e6a7-4d88-9e6f-8589daf5a2db admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:28.025562 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920d2ac-e6a7-4d88-9e6f-8589daf5a2db admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:28.030384 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c262e-230d-41a3-8ff0-01e8024baf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:28.031050 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c262e-230d-41a3-8ff0-01e8024baf7f admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:28.031337 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c262e-230d-41a3-8ff0-01e8024baf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:28.031617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c262e-230d-41a3-8ff0-01e8024baf7f admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:28.032388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1c262e-230d-41a3-8ff0-01e8024baf7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:28.034027 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1c262e-230d-41a3-8ff0-01e8024baf7f admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:28.044729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1401/2841] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:35:28.048759 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85720f-159e-45de-afcf-205686e51e6e admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:28.058894 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85720f-159e-45de-afcf-205686e51e6e admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:28.063520 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c6753-ca5e-4692-8e26-cedf7eab5ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:28.064215 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c6753-ca5e-4692-8e26-cedf7eab5ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:28.064502 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c6753-ca5e-4692-8e26-cedf7eab5ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:28.065274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c6753-ca5e-4692-8e26-cedf7eab5ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:28.068323 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c6753-ca5e-4692-8e26-cedf7eab5ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:28.068323 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c6753-ca5e-4692-8e26-cedf7eab5ef6 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:28.077744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1441/2842] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:35:28.082497 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c262e-230d-41a3-8ff0-01e8024baf7f admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:28.102239 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c262e-230d-41a3-8ff0-01e8024baf7f admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:28.107248 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbac31d-161d-4a62-b7b7-ee0f02f6ef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:28.108374 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbac31d-161d-4a62-b7b7-ee0f02f6ef43 admin admin] REQUEST_METHOD: `PUT` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:28.108661 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbac31d-161d-4a62-b7b7-ee0f02f6ef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:28.108936 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbac31d-161d-4a62-b7b7-ee0f02f6ef43 admin admin] PATH_INFO: `/v3/projects/262ae6d900ad4cc7afa231a5f271935b/users/5bbb0e5de132476189fcd28284fa330e/roles/d8b94102152a46768d568f66794854f3` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:28.124980 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbac31d-161d-4a62-b7b7-ee0f02f6ef43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=262ae6d900ad4cc7afa231a5f271935b, user_id=5bbb0e5de132476189fcd28284fa330e, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:28.128256 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbac31d-161d-4a62-b7b7-ee0f02f6ef43 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:28.141992 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1402/2843] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:35:28 2022] PUT /identity/v3/projects/262ae6d900ad4cc7afa231a5f271935b/users/5bbb0e5de132476189fcd28284fa330e/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 04:35:28.152263 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8cefe-9e3b-435a-9a65-78921bb0fcce None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:28.152553 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8cefe-9e3b-435a-9a65-78921bb0fcce None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:28.152833 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8cefe-9e3b-435a-9a65-78921bb0fcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:28.180122 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-bcb8cefe-9e3b-435a-9a65-78921bb0fcce None None] MFA Rules not processed for user `5bbb0e5de132476189fcd28284fa330e`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:28.181344 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-bcb8cefe-9e3b-435a-9a65-78921bb0fcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:28.247509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1442/2844] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:28.258709 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbac31d-161d-4a62-b7b7-ee0f02f6ef43 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:28.260424 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-5bbac31d-161d-4a62-b7b7-ee0f02f6ef43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:28.315195 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbac31d-161d-4a62-b7b7-ee0f02f6ef43 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:28.319315 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453ed95-ef69-4981-987e-dc6d7829fbcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:28.319986 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453ed95-ef69-4981-987e-dc6d7829fbcc service cinder] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:28.320268 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453ed95-ef69-4981-987e-dc6d7829fbcc service cinder] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:28.320540 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453ed95-ef69-4981-987e-dc6d7829fbcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:28.328737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453ed95-ef69-4981-987e-dc6d7829fbcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:28.330204 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453ed95-ef69-4981-987e-dc6d7829fbcc service cinder] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:28.338916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1403/2845] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:35:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:35:30.769602 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f70ec5-bf27-4f24-8aee-29482ae6778c None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:30.770076 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f70ec5-bf27-4f24-8aee-29482ae6778c None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:30.770210 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f70ec5-bf27-4f24-8aee-29482ae6778c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:30.787091 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-d3f70ec5-bf27-4f24-8aee-29482ae6778c None None] MFA Rules not processed for user `5bbb0e5de132476189fcd28284fa330e`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:30.788343 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d3f70ec5-bf27-4f24-8aee-29482ae6778c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:30.795138 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1443/2846] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:44.528941 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48729d7c-1ec9-445d-8d5c-f9ce12e019bf None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:44.529410 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48729d7c-1ec9-445d-8d5c-f9ce12e019bf None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:44.529520 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48729d7c-1ec9-445d-8d5c-f9ce12e019bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:44.547161 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-48729d7c-1ec9-445d-8d5c-f9ce12e019bf None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:44.548379 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-48729d7c-1ec9-445d-8d5c-f9ce12e019bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:44.602325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1404/2847] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:35:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:44.614214 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f70ec5-bf27-4f24-8aee-29482ae6778c None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:44.615569 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-d3f70ec5-bf27-4f24-8aee-29482ae6778c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:44.682426 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f70ec5-bf27-4f24-8aee-29482ae6778c None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:44.687223 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b209f-8b13-46f6-b86a-9f1aefdbdb49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:44.687862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b209f-8b13-46f6-b86a-9f1aefdbdb49 service swift] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:44.688184 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b209f-8b13-46f6-b86a-9f1aefdbdb49 service swift] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:44.688453 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b209f-8b13-46f6-b86a-9f1aefdbdb49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:44.696373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b209f-8b13-46f6-b86a-9f1aefdbdb49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:44.697925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b209f-8b13-46f6-b86a-9f1aefdbdb49 service swift] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:44.705618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1444/2848] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:35:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:35:45.201340 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6900937-de8f-49de-9ca6-38667a7845f8 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:45.201637 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6900937-de8f-49de-9ca6-38667a7845f8 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:45.201917 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6900937-de8f-49de-9ca6-38667a7845f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:45.219611 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-d6900937-de8f-49de-9ca6-38667a7845f8 None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:45.220832 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d6900937-de8f-49de-9ca6-38667a7845f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:45.228703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1405/2849] 10.222.0.44 () {58 vars in 967 bytes} [Mon May 9 04:35:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:45.241550 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b209f-8b13-46f6-b86a-9f1aefdbdb49 service swift] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:45.251773 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b209f-8b13-46f6-b86a-9f1aefdbdb49 service swift] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:45.256804 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5005f5-bb73-4037-891d-3eecf642bad0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:45.257668 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5005f5-bb73-4037-891d-3eecf642bad0 service swift] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:45.257948 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5005f5-bb73-4037-891d-3eecf642bad0 service swift] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:45.258220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5005f5-bb73-4037-891d-3eecf642bad0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:45.266199 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5005f5-bb73-4037-891d-3eecf642bad0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:45.267751 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5005f5-bb73-4037-891d-3eecf642bad0 service swift] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:45.276114 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1445/2850] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:35:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:35:45.458234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df22fed-908a-4744-a263-c366756bdd29 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:45.458522 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df22fed-908a-4744-a263-c366756bdd29 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:45.458792 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df22fed-908a-4744-a263-c366756bdd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:45.476915 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-8df22fed-908a-4744-a263-c366756bdd29 None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:45.478080 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-8df22fed-908a-4744-a263-c366756bdd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:45.484977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1406/2851] 10.222.0.44 () {58 vars in 1009 bytes} [Mon May 9 04:35:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:45.496849 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5005f5-bb73-4037-891d-3eecf642bad0 service swift] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:45.507558 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5005f5-bb73-4037-891d-3eecf642bad0 service swift] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:45.512482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca4bd7-48e8-4238-96ef-24489cba33ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:45.513182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca4bd7-48e8-4238-96ef-24489cba33ad service swift] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:45.513470 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca4bd7-48e8-4238-96ef-24489cba33ad service swift] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:45.513747 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca4bd7-48e8-4238-96ef-24489cba33ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:45.522085 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca4bd7-48e8-4238-96ef-24489cba33ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:45.523650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca4bd7-48e8-4238-96ef-24489cba33ad service swift] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:45.531724 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1446/2852] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:35:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:35:45.834471 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1c0fb-49ea-4af0-bb97-b738eb565f64 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:45.834928 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1c0fb-49ea-4af0-bb97-b738eb565f64 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:45.835040 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1c0fb-49ea-4af0-bb97-b738eb565f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:45.852592 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-25c1c0fb-49ea-4af0-bb97-b738eb565f64 None None] MFA Rules not processed for user `ee0055c75b91471a99d9bb18248839ea`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:45.853864 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-25c1c0fb-49ea-4af0-bb97-b738eb565f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:45.862471 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1407/2853] 10.222.0.44 () {58 vars in 967 bytes} [Mon May 9 04:35:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:45.874822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca4bd7-48e8-4238-96ef-24489cba33ad service swift] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:45.885613 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca4bd7-48e8-4238-96ef-24489cba33ad service swift] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:45.890532 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe6a6e-e141-4fc5-a2fb-6b820b066698 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c46e7cc6407f44f08a659288e3ad0767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:45.891160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe6a6e-e141-4fc5-a2fb-6b820b066698 service swift] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:45.891439 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe6a6e-e141-4fc5-a2fb-6b820b066698 service swift] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:45.891711 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe6a6e-e141-4fc5-a2fb-6b820b066698 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:45.900306 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe6a6e-e141-4fc5-a2fb-6b820b066698 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:45.901985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe6a6e-e141-4fc5-a2fb-6b820b066698 service swift] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:45.909663 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1447/2854] 10.222.0.44 () {60 vars in 1384 bytes} [Mon May 9 04:35:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 04:35:47.915435 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1c0fb-49ea-4af0-bb97-b738eb565f64 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:47.976859 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1c0fb-49ea-4af0-bb97-b738eb565f64 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:47.981312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:47.982173 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:47.982455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:47.982729 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] PATH_INFO: `/v3/users/5bbb0e5de132476189fcd28284fa330e` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:47.983977 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bbb0e5de132476189fcd28284fa330e)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:47.985803 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:48.029032 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bbb0e5de132476189fcd28284fa330e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:48.029713 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bbb0e5de132476189fcd28284fa330e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:48.030312 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bbb0e5de132476189fcd28284fa330e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:48.037566 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bbb0e5de132476189fcd28284fa330e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:48.044500 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bbb0e5de132476189fcd28284fa330e'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:48.061388 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1408/2855] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:35:47 2022] DELETE /identity/v3/users/5bbb0e5de132476189fcd28284fa330e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:35:48.065826 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319c359-f4d4-4bd2-8bf0-1f5a93da837e None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:48.066327 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319c359-f4d4-4bd2-8bf0-1f5a93da837e None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:48.066447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319c359-f4d4-4bd2-8bf0-1f5a93da837e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:48.083151 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-9319c359-f4d4-4bd2-8bf0-1f5a93da837e None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:48.084352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-9319c359-f4d4-4bd2-8bf0-1f5a93da837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:48.145041 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1448/2856] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:48.154650 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:48.156095 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:48.211037 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dc54a-e183-4d3c-9ccf-fddec0c3bfc6 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:48.214632 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5497a-adcf-4095-ba6b-b8d4c28841dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:48.215517 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5497a-adcf-4095-ba6b-b8d4c28841dc service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:48.215795 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5497a-adcf-4095-ba6b-b8d4c28841dc service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:48.216120 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5497a-adcf-4095-ba6b-b8d4c28841dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:48.225452 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5497a-adcf-4095-ba6b-b8d4c28841dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:48.226866 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5497a-adcf-4095-ba6b-b8d4c28841dc service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:48.236127 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1409/2857] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:35:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:35:48.401875 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319c359-f4d4-4bd2-8bf0-1f5a93da837e None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:48.403274 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-9319c359-f4d4-4bd2-8bf0-1f5a93da837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:48.418186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319c359-f4d4-4bd2-8bf0-1f5a93da837e None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:48.423220 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:48.424139 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:48.424595 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:48.424882 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd admin admin] PATH_INFO: `/v3/projects/262ae6d900ad4cc7afa231a5f271935b` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:48.426154 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=262ae6d900ad4cc7afa231a5f271935b)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:48.428174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:48.473775 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '262ae6d900ad4cc7afa231a5f271935b', 'request_id': 'req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:48.478906 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '262ae6d900ad4cc7afa231a5f271935b', 'request_id': 'req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:48.485636 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '262ae6d900ad4cc7afa231a5f271935b', 'request_id': 'req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:48.496455 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd admin admin] The token cache is being invalidate because project 262ae6d900ad4cc7afa231a5f271935b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:35:48.496758 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:48.497403 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-8a42edbe-3b49-4bc3-baea-b76c61bbd0fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:48.498945 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1449/2858] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:35:48 2022] DELETE /identity/v3/projects/262ae6d900ad4cc7afa231a5f271935b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:35:57.290659 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5497a-adcf-4095-ba6b-b8d4c28841dc service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:57.292158 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-d3c5497a-adcf-4095-ba6b-b8d4c28841dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:57.305263 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5497a-adcf-4095-ba6b-b8d4c28841dc service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:57.358711 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:57.359577 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:57.359866 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:57.360171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] PATH_INFO: `/v3/users/56f5b7db02674c6f80ef61d2410e9208` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:57.361338 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] RBAC: Authorizing `identity:delete_user(user_id=56f5b7db02674c6f80ef61d2410e9208)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:57.363741 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:57.406577 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56f5b7db02674c6f80ef61d2410e9208'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:57.407213 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56f5b7db02674c6f80ef61d2410e9208'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:57.407819 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56f5b7db02674c6f80ef61d2410e9208'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:57.414523 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56f5b7db02674c6f80ef61d2410e9208'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:57.421094 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56f5b7db02674c6f80ef61d2410e9208'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:57.438436 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1410/2859] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:35:57 2022] DELETE /identity/v3/users/56f5b7db02674c6f80ef61d2410e9208 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:35:57.443537 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b0420-10c0-4419-87b1-5169ffa07244 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:57.444053 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b0420-10c0-4419-87b1-5169ffa07244 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:57.444243 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b0420-10c0-4419-87b1-5169ffa07244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:57.461644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-132b0420-10c0-4419-87b1-5169ffa07244 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:57.462854 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-132b0420-10c0-4419-87b1-5169ffa07244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:57.518888 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1450/2860] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:57.528824 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:57.530200 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:57.585479 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a87dba-39f4-4c77-bd14-0e9a5c14938b admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:57.589067 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e69d1-8360-4d41-a5fa-45dc6ebaffb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:57.589736 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e69d1-8360-4d41-a5fa-45dc6ebaffb6 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:57.590019 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e69d1-8360-4d41-a5fa-45dc6ebaffb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:57.590557 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e69d1-8360-4d41-a5fa-45dc6ebaffb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:57.599471 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e69d1-8360-4d41-a5fa-45dc6ebaffb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:57.600895 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e69d1-8360-4d41-a5fa-45dc6ebaffb6 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:57.610011 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1411/2861] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:35:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:35:57.874552 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b0420-10c0-4419-87b1-5169ffa07244 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:57.876097 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-132b0420-10c0-4419-87b1-5169ffa07244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:57.889432 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b0420-10c0-4419-87b1-5169ffa07244 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:57.895160 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcf359-c35f-4126-9293-e733bf8cae60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:57.896756 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcf359-c35f-4126-9293-e733bf8cae60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:57.897208 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcf359-c35f-4126-9293-e733bf8cae60 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:57.897484 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcf359-c35f-4126-9293-e733bf8cae60 admin admin] PATH_INFO: `/v3/projects/62bb515e67e64fa99ed96a95b5aee9b8` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:57.898707 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcf359-c35f-4126-9293-e733bf8cae60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62bb515e67e64fa99ed96a95b5aee9b8)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:57.900615 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcf359-c35f-4126-9293-e733bf8cae60 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:57.947040 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-06bcf359-c35f-4126-9293-e733bf8cae60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62bb515e67e64fa99ed96a95b5aee9b8', 'request_id': 'req-06bcf359-c35f-4126-9293-e733bf8cae60'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:57.952192 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-06bcf359-c35f-4126-9293-e733bf8cae60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62bb515e67e64fa99ed96a95b5aee9b8', 'request_id': 'req-06bcf359-c35f-4126-9293-e733bf8cae60'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:57.959619 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-06bcf359-c35f-4126-9293-e733bf8cae60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62bb515e67e64fa99ed96a95b5aee9b8', 'request_id': 'req-06bcf359-c35f-4126-9293-e733bf8cae60'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:57.970366 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-06bcf359-c35f-4126-9293-e733bf8cae60 admin admin] The token cache is being invalidate because project 62bb515e67e64fa99ed96a95b5aee9b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:35:57.970645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-06bcf359-c35f-4126-9293-e733bf8cae60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:57.971252 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-06bcf359-c35f-4126-9293-e733bf8cae60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:35:57.972862 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1451/2862] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:35:57 2022] DELETE /identity/v3/projects/62bb515e67e64fa99ed96a95b5aee9b8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:35:57.977747 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca835b-0cd5-4e91-9ee6-15694705f2b7 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:57.978038 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca835b-0cd5-4e91-9ee6-15694705f2b7 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:57.978319 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca835b-0cd5-4e91-9ee6-15694705f2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:57.995001 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-dcca835b-0cd5-4e91-9ee6-15694705f2b7 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:57.996222 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-dcca835b-0cd5-4e91-9ee6-15694705f2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:58.052737 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1412/2863] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:58.061137 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b25ecb-b86e-4ae3-bcba-fa30cb644b9b None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:58.061428 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b25ecb-b86e-4ae3-bcba-fa30cb644b9b None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:58.061703 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b25ecb-b86e-4ae3-bcba-fa30cb644b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:58.079745 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-c5b25ecb-b86e-4ae3-bcba-fa30cb644b9b None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:58.080936 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-c5b25ecb-b86e-4ae3-bcba-fa30cb644b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:58.088720 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1452/2864] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:58.093070 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca835b-0cd5-4e91-9ee6-15694705f2b7 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:58.102628 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca835b-0cd5-4e91-9ee6-15694705f2b7 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:58.107112 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c7c12-55c7-412c-b515-a546a67ed1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:58.107788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c7c12-55c7-412c-b515-a546a67ed1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:58.108085 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c7c12-55c7-412c-b515-a546a67ed1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:58.108365 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c7c12-55c7-412c-b515-a546a67ed1c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:58.109087 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c7c12-55c7-412c-b515-a546a67ed1c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=79763) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 04:35:58.109395 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c7c12-55c7-412c-b515-a546a67ed1c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:58.110982 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c7c12-55c7-412c-b515-a546a67ed1c4 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:58.126253 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1413/2865] 10.222.0.44 () {58 vars in 1152 bytes} [Mon May 9 04:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 04:35:58.130954 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b25ecb-b86e-4ae3-bcba-fa30cb644b9b None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:58.142863 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b25ecb-b86e-4ae3-bcba-fa30cb644b9b None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:58.147831 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91270b2a-9c4f-4ccf-be73-f8947fc0e18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:58.148550 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91270b2a-9c4f-4ccf-be73-f8947fc0e18c admin admin] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:58.148833 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91270b2a-9c4f-4ccf-be73-f8947fc0e18c admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:58.149107 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91270b2a-9c4f-4ccf-be73-f8947fc0e18c admin admin] PATH_INFO: `/v3/projects` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:58.149801 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91270b2a-9c4f-4ccf-be73-f8947fc0e18c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:58.151719 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91270b2a-9c4f-4ccf-be73-f8947fc0e18c admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:58.177776 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1453/2866] 10.222.0.44 () {60 vars in 1152 bytes} [Mon May 9 04:35:58 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:35:58.182172 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c7c12-55c7-412c-b515-a546a67ed1c4 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:58.193883 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c7c12-55c7-412c-b515-a546a67ed1c4 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:58.252351 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef5848-5aa2-4f31-8096-ba02ded60723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:58.253021 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef5848-5aa2-4f31-8096-ba02ded60723 admin admin] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:58.253325 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef5848-5aa2-4f31-8096-ba02ded60723 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:58.254278 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef5848-5aa2-4f31-8096-ba02ded60723 admin admin] PATH_INFO: `/v3/users` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:58.254960 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef5848-5aa2-4f31-8096-ba02ded60723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:58.256904 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef5848-5aa2-4f31-8096-ba02ded60723 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:58.275551 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1414/2867] 10.222.0.44 () {60 vars in 1143 bytes} [Mon May 9 04:35:58 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 04:35:58.280827 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91270b2a-9c4f-4ccf-be73-f8947fc0e18c admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:58.290696 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91270b2a-9c4f-4ccf-be73-f8947fc0e18c admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:58.295368 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d570bb-ccf7-4870-b80d-32d6801f7e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:58.296063 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d570bb-ccf7-4870-b80d-32d6801f7e93 admin admin] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:58.296340 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d570bb-ccf7-4870-b80d-32d6801f7e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:58.296610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d570bb-ccf7-4870-b80d-32d6801f7e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:58.297326 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d570bb-ccf7-4870-b80d-32d6801f7e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:58.299061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d570bb-ccf7-4870-b80d-32d6801f7e93 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:58.311390 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1454/2868] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:35:58.315271 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef5848-5aa2-4f31-8096-ba02ded60723 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:58.326561 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef5848-5aa2-4f31-8096-ba02ded60723 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:58.331370 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f34a8-d2d6-47b1-b419-2a0666419719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:58.332251 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f34a8-d2d6-47b1-b419-2a0666419719 admin admin] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:58.332610 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f34a8-d2d6-47b1-b419-2a0666419719 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:58.332892 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f34a8-d2d6-47b1-b419-2a0666419719 admin admin] PATH_INFO: `/v3/roles` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:58.333615 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f34a8-d2d6-47b1-b419-2a0666419719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:58.335455 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f34a8-d2d6-47b1-b419-2a0666419719 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:58.346014 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1415/2869] 10.222.0.44 () {58 vars in 1121 bytes} [Mon May 9 04:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 04:35:58.350512 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d570bb-ccf7-4870-b80d-32d6801f7e93 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:58.360494 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d570bb-ccf7-4870-b80d-32d6801f7e93 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:58.366610 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924da388-7aef-449e-8098-5209f2932694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:58.367511 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924da388-7aef-449e-8098-5209f2932694 admin admin] REQUEST_METHOD: `PUT` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:58.367777 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924da388-7aef-449e-8098-5209f2932694 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:58.368147 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924da388-7aef-449e-8098-5209f2932694 admin admin] PATH_INFO: `/v3/projects/ec208afbaa5b4b60a9c7929cee44f32a/users/a57df1959631472db71aad25bc42f8ce/roles/d8b94102152a46768d568f66794854f3` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:58.383531 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924da388-7aef-449e-8098-5209f2932694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec208afbaa5b4b60a9c7929cee44f32a, user_id=a57df1959631472db71aad25bc42f8ce, role_id=d8b94102152a46768d568f66794854f3)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:58.386183 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924da388-7aef-449e-8098-5209f2932694 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:58.399108 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1455/2870] 10.222.0.44 () {60 vars in 1482 bytes} [Mon May 9 04:35:58 2022] PUT /identity/v3/projects/ec208afbaa5b4b60a9c7929cee44f32a/users/a57df1959631472db71aad25bc42f8ce/roles/d8b94102152a46768d568f66794854f3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:35:58.409244 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0781bf3-ce84-426f-9cec-6eed2c3d14d5 None None] REQUEST_METHOD: `POST` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:58.409582 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0781bf3-ce84-426f-9cec-6eed2c3d14d5 None None] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:58.409976 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0781bf3-ce84-426f-9cec-6eed2c3d14d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:58.436250 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.auth.core [None req-c0781bf3-ce84-426f-9cec-6eed2c3d14d5 None None] MFA Rules not processed for user `a57df1959631472db71aad25bc42f8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=79763) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:35:58.437457 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-c0781bf3-ce84-426f-9cec-6eed2c3d14d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:58.498749 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1416/2871] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:35:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:35:58.510099 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924da388-7aef-449e-8098-5209f2932694 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:35:58.511865 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-924da388-7aef-449e-8098-5209f2932694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:35:58.567414 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924da388-7aef-449e-8098-5209f2932694 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:35:58.570942 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f048422-ba7d-40b8-8dbf-85fe671888b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72caf90f411f495bb22aa85df4d25eb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:35:58.571586 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f048422-ba7d-40b8-8dbf-85fe671888b4 service cinder] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:35:58.571859 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f048422-ba7d-40b8-8dbf-85fe671888b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:35:58.572163 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f048422-ba7d-40b8-8dbf-85fe671888b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:35:58.580064 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f048422-ba7d-40b8-8dbf-85fe671888b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:35:58.581500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f048422-ba7d-40b8-8dbf-85fe671888b4 service cinder] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:35:58.589841 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1456/2872] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:35:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:36:06.577066 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0781bf3-ce84-426f-9cec-6eed2c3d14d5 None None] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:36:06.637245 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0781bf3-ce84-426f-9cec-6eed2c3d14d5 None None] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:36:06.641664 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:36:06.642519 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:36:06.642802 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:36:06.643076 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] PATH_INFO: `/v3/users/a57df1959631472db71aad25bc42f8ce` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:36:06.644261 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a57df1959631472db71aad25bc42f8ce)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:36:06.646162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:36:06.692774 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a57df1959631472db71aad25bc42f8ce'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:06.693415 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a57df1959631472db71aad25bc42f8ce'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:06.694007 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a57df1959631472db71aad25bc42f8ce'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:06.700889 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a57df1959631472db71aad25bc42f8ce'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:06.707427 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a57df1959631472db71aad25bc42f8ce'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:06.724181 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1417/2873] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:36:06 2022] DELETE /identity/v3/users/a57df1959631472db71aad25bc42f8ce => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:36:06.728570 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff674ff-da2d-4d76-818a-61b29b62de06 None None] REQUEST_METHOD: `POST` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:36:06.729057 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff674ff-da2d-4d76-818a-61b29b62de06 None None] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:36:06.729156 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff674ff-da2d-4d76-818a-61b29b62de06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:36:06.746961 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.auth.core [None req-2ff674ff-da2d-4d76-818a-61b29b62de06 None None] MFA Rules not processed for user `c49606bc956845408ea8db21c53f217a`. Rule list: `[]` (Enabled: `True`). {{(pid=79764) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 04:36:06.748186 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2ff674ff-da2d-4d76-818a-61b29b62de06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:06.803403 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1457/2874] 10.222.0.44 () {58 vars in 957 bytes} [Mon May 9 04:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 04:36:06.813376 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:36:06.814739 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:06.872121 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252b8f5-729e-4af7-b434-2d09d3f842c0 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:36:06.875723 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f574c-0627-4791-8ac1-35b391f8d9f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:36:06.876397 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f574c-0627-4791-8ac1-35b391f8d9f8 service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:36:06.876679 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f574c-0627-4791-8ac1-35b391f8d9f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:36:06.876952 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f574c-0627-4791-8ac1-35b391f8d9f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:36:06.885617 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f574c-0627-4791-8ac1-35b391f8d9f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:36:06.887018 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f574c-0627-4791-8ac1-35b391f8d9f8 service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:36:06.896199 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1418/2875] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:36:07.173783 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff674ff-da2d-4d76-818a-61b29b62de06 None None] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:36:07.175483 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2ff674ff-da2d-4d76-818a-61b29b62de06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:07.188667 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff674ff-da2d-4d76-818a-61b29b62de06 None None] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:36:07.194009 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:36:07.195001 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:36:07.195270 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:36:07.195542 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] PATH_INFO: `/v3/projects/ec208afbaa5b4b60a9c7929cee44f32a` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:36:07.196891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec208afbaa5b4b60a9c7929cee44f32a)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:36:07.198822 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:36:07.247555 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec208afbaa5b4b60a9c7929cee44f32a', 'request_id': 'req-859f01aa-396a-4c05-9cb8-dc0bd363f713'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:07.253546 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec208afbaa5b4b60a9c7929cee44f32a', 'request_id': 'req-859f01aa-396a-4c05-9cb8-dc0bd363f713'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:07.261169 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec208afbaa5b4b60a9c7929cee44f32a', 'request_id': 'req-859f01aa-396a-4c05-9cb8-dc0bd363f713'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:07.272349 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] The token cache is being invalidate because project ec208afbaa5b4b60a9c7929cee44f32a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:36:07.272618 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:07.273234 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:07.274748 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1458/2876] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:36:07 2022] DELETE /identity/v3/projects/ec208afbaa5b4b60a9c7929cee44f32a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:36:08.558133 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f574c-0627-4791-8ac1-35b391f8d9f8 service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:36:08.560197 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-232f574c-0627-4791-8ac1-35b391f8d9f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:08.573371 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f574c-0627-4791-8ac1-35b391f8d9f8 service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:36:08.626425 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:36:08.627217 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:36:08.627496 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:36:08.627768 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] PATH_INFO: `/v3/users/ec0360cd80a040c5ae2f3f7a6ba01f75` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:36:08.629013 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec0360cd80a040c5ae2f3f7a6ba01f75)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:36:08.630897 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:36:08.674375 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec0360cd80a040c5ae2f3f7a6ba01f75'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:08.675063 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec0360cd80a040c5ae2f3f7a6ba01f75'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:08.675706 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec0360cd80a040c5ae2f3f7a6ba01f75'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:08.682785 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec0360cd80a040c5ae2f3f7a6ba01f75'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:08.689321 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec0360cd80a040c5ae2f3f7a6ba01f75'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:08.705576 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1419/2877] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:36:08 2022] DELETE /identity/v3/users/ec0360cd80a040c5ae2f3f7a6ba01f75 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:36:08.876996 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:36:08.878917 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:08.895373 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f01aa-396a-4c05-9cb8-dc0bd363f713 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:36:08.953987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:36:08.954891 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:36:08.955174 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:36:08.955447 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] PATH_INFO: `/v3/projects/3d6aeac77b6d4f1c8f912d7c5d91968f` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:36:08.956650 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d6aeac77b6d4f1c8f912d7c5d91968f)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:36:08.958717 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:36:09.003726 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d6aeac77b6d4f1c8f912d7c5d91968f', 'request_id': 'req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:09.009090 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d6aeac77b6d4f1c8f912d7c5d91968f', 'request_id': 'req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:09.015988 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d6aeac77b6d4f1c8f912d7c5d91968f', 'request_id': 'req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:09.027235 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] The token cache is being invalidate because project 3d6aeac77b6d4f1c8f912d7c5d91968f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79764) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:36:09.027524 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:09.028182 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:09.029734 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1459/2878] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:36:08 2022] DELETE /identity/v3/projects/3d6aeac77b6d4f1c8f912d7c5d91968f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:36:19.224032 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:36:19.226162 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:19.281903 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481676f1-0135-423d-a285-b108685a3387 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:36:19.329299 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a482afa-c70f-417f-b3d0-b1ff6b88ae6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:36:19.329939 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482afa-c70f-417f-b3d0-b1ff6b88ae6e service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:36:19.330218 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482afa-c70f-417f-b3d0-b1ff6b88ae6e service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:36:19.330490 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482afa-c70f-417f-b3d0-b1ff6b88ae6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:36:19.332277 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1a482afa-c70f-417f-b3d0-b1ff6b88ae6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:19.400632 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a482afa-c70f-417f-b3d0-b1ff6b88ae6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:36:19.402056 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a482afa-c70f-417f-b3d0-b1ff6b88ae6e service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:36:19.414012 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1420/2879] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:36:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:36:20.591127 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:36:20.592744 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:20.605787 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07bdc1-9738-4ff0-a8b3-a42c535985d4 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:36:20.609386 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df24e5-48c1-42d9-a743-78cf58f37998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:36:20.610100 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df24e5-48c1-42d9-a743-78cf58f37998 service neutron] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:36:20.610388 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df24e5-48c1-42d9-a743-78cf58f37998 service neutron] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:36:20.610657 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df24e5-48c1-42d9-a743-78cf58f37998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:36:20.612591 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a2df24e5-48c1-42d9-a743-78cf58f37998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:20.626051 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df24e5-48c1-42d9-a743-78cf58f37998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:36:20.627500 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df24e5-48c1-42d9-a743-78cf58f37998 service neutron] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:36:20.639176 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1460/2880] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:36:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:36:23.628484 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a482afa-c70f-417f-b3d0-b1ff6b88ae6e service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:36:23.629991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1a482afa-c70f-417f-b3d0-b1ff6b88ae6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:23.640570 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a482afa-c70f-417f-b3d0-b1ff6b88ae6e service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:36:23.644793 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd54a8-af09-4d5c-87cd-57909f397caf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d852975a2e84925a45775a16cc52f32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:36:23.645430 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd54a8-af09-4d5c-87cd-57909f397caf service neutron] REQUEST_METHOD: `GET` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:36:23.645715 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd54a8-af09-4d5c-87cd-57909f397caf service neutron] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:36:23.645991 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd54a8-af09-4d5c-87cd-57909f397caf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:36:23.647675 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-1bfd54a8-af09-4d5c-87cd-57909f397caf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:23.709605 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd54a8-af09-4d5c-87cd-57909f397caf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:36:23.710987 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd54a8-af09-4d5c-87cd-57909f397caf service neutron] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:36:23.720924 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1421/2881] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 04:36:30.082916 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df24e5-48c1-42d9-a743-78cf58f37998 service neutron] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:36:30.084525 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-a2df24e5-48c1-42d9-a743-78cf58f37998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:30.100987 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df24e5-48c1-42d9-a743-78cf58f37998 service neutron] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:36:30.105752 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:36:30.106644 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:36:30.106925 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:36:30.107352 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] PATH_INFO: `/v3/users/cb34feb512764ec5b9ca25c30c3d7159` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:36:30.108682 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb34feb512764ec5b9ca25c30c3d7159)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:36:30.110609 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:36:30.157974 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb34feb512764ec5b9ca25c30c3d7159'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.158645 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb34feb512764ec5b9ca25c30c3d7159'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.166482 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb34feb512764ec5b9ca25c30c3d7159'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.173238 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb34feb512764ec5b9ca25c30c3d7159'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.173881 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb34feb512764ec5b9ca25c30c3d7159'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.190584 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1461/2882] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:36:30 2022] DELETE /identity/v3/users/cb34feb512764ec5b9ca25c30c3d7159 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:36:30.363703 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd54a8-af09-4d5c-87cd-57909f397caf service neutron] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:36:30.374502 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd54a8-af09-4d5c-87cd-57909f397caf service neutron] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:36:30.430788 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:36:30.431647 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:36:30.431956 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:36:30.432234 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] PATH_INFO: `/v3/projects/b6223852af9c4361b99a24e7a49a7734` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:36:30.433385 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6223852af9c4361b99a24e7a49a7734)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:36:30.435255 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:36:30.480402 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6223852af9c4361b99a24e7a49a7734', 'request_id': 'req-46c1d09e-3dfb-418a-a09b-872e5f0852b2'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.486344 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6223852af9c4361b99a24e7a49a7734', 'request_id': 'req-46c1d09e-3dfb-418a-a09b-872e5f0852b2'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.491711 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6223852af9c4361b99a24e7a49a7734', 'request_id': 'req-46c1d09e-3dfb-418a-a09b-872e5f0852b2'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.503083 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] The token cache is being invalidate because project b6223852af9c4361b99a24e7a49a7734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:36:30.503364 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.504005 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.505539 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1422/2883] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:36:30 2022] DELETE /identity/v3/projects/b6223852af9c4361b99a24e7a49a7734 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:36:30.509580 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:36:30.511061 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:30.524350 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5162f16-9213-4805-98c7-c528f09652a0 admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:36:30.578121 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:36:30.578992 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:36:30.579277 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:36:30.579554 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] PATH_INFO: `/v3/users/26beba5551a94841a0c022a02e71aaa6` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:36:30.580767 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=26beba5551a94841a0c022a02e71aaa6)` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:36:30.583246 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:36:30.626312 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26beba5551a94841a0c022a02e71aaa6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.626877 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26beba5551a94841a0c022a02e71aaa6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.633683 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26beba5551a94841a0c022a02e71aaa6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.640204 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26beba5551a94841a0c022a02e71aaa6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.640826 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.notifications [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26beba5551a94841a0c022a02e71aaa6'} {{(pid=79764) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.657509 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1462/2884] 10.222.0.44 () {58 vars in 1223 bytes} [Mon May 9 04:36:30 2022] DELETE /identity/v3/users/26beba5551a94841a0c022a02e71aaa6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:36:30.827997 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] Authenticating user token {{(pid=79763) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:36:30.829476 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.fernet_utils [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79763) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:36:30.843582 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1d09e-3dfb-418a-a09b-872e5f0852b2 admin admin] Validating token access rules against request {{(pid=79763) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:36:30.900916 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c49606bc956845408ea8db21c53f217a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '528420bd736843dc87d64954f4596bbd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79763) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:36:30.901707 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:36:30.901993 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea admin admin] SCRIPT_NAME: `/identity` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:36:30.902272 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea admin admin] PATH_INFO: `/v3/projects/c54b5ee463964fa289d41b32b9d47efa` {{(pid=79763) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:36:30.903444 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=c54b5ee463964fa289d41b32b9d47efa)` {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:36:30.905374 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea admin admin] RBAC: Authorization granted {{(pid=79763) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:36:30.954814 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c54b5ee463964fa289d41b32b9d47efa', 'request_id': 'req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.961488 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c54b5ee463964fa289d41b32b9d47efa', 'request_id': 'req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.967024 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c54b5ee463964fa289d41b32b9d47efa', 'request_id': 'req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea'} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.978636 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea admin admin] The token cache is being invalidate because project c54b5ee463964fa289d41b32b9d47efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=79763) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 09 04:36:30.978926 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.979542 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: DEBUG keystone.notifications [None req-b5c5b121-2af8-4bfd-aed4-57fadd7744ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=79763) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 09 04:36:30.981171 dv-cinder-838009-3-iscsi devstack@keystone.service[79763]: [pid: 79763|app: 0|req: 1423/2885] 10.222.0.44 () {58 vars in 1232 bytes} [Mon May 9 04:36:30 2022] DELETE /identity/v3/projects/c54b5ee463964fa289d41b32b9d47efa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 04:40:28.654647 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] Authenticating user token {{(pid=79764) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 04:40:28.656144 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:40:28.788475 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab1b09-1a17-4262-b108-51f05a3bfc7d admin admin] Validating token access rules against request {{(pid=79764) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 04:40:28.845093 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4e979-466d-4dc2-aab3-4a6c1ae6a6da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd41e05b7ef954dd596c0134b65434dd6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3322b00723974bd99e837d447bac22d1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=79764) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 04:40:28.845743 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e979-466d-4dc2-aab3-4a6c1ae6a6da service placement] REQUEST_METHOD: `GET` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 04:40:28.846027 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e979-466d-4dc2-aab3-4a6c1ae6a6da service placement] SCRIPT_NAME: `/identity` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 04:40:28.846302 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e979-466d-4dc2-aab3-4a6c1ae6a6da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=79764) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 04:40:28.848089 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.fernet_utils [None req-83f4e979-466d-4dc2-aab3-4a6c1ae6a6da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=79764) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 04:40:28.859922 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4e979-466d-4dc2-aab3-4a6c1ae6a6da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 04:40:28.861503 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4e979-466d-4dc2-aab3-4a6c1ae6a6da service placement] RBAC: Authorization granted {{(pid=79764) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 04:40:28.870985 dv-cinder-838009-3-iscsi devstack@keystone.service[79764]: [pid: 79764|app: 0|req: 1463/2886] 10.222.0.44 () {60 vars in 1365 bytes} [Mon May 9 04:40:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)